Streaming PCM audio over tcp socket - ios

Question

I have a continuous stream of raw PCM audio data from a TCP socket and I want to play them. I've done so many researches and saw many samples but no result. This gist was the most close solution but the problem is, it's streaming mp3 file.
So I have a socket which receives linear PCM audio data and give them to the player like this:
func play(_ data: Data) {
// this function is called for every 320 bytes of linear PCM data.
// play the 320 bytes of PCM data here!
}
So is there any "Simple" way to play raw PCM audio data?

Answer 1

For iOS, you can use the RemoteIO Audio Unit or the AVAudioEngine with a circular buffer for real-time audio streaming.
You can't give network data directly to audio output, but instead should put it in a circular buffer from which an audio subsystem play callback can consume it at its fixed rate. You will need to pre-buffer some amount of audio samples to cover network jitter.
Simple "ways" of doing this might not handle network jitter gracefully.

Answer 2

Answering late but If you are still stuck in playing TCP bytes then try to follow my answer where you put your tcp audio bytes in Circular Buffer and play it via AudioUnit.


Below code receives bytes from TCP and put them into a TPCircularBuffer

func tcpReceive() {
receivingQueue.async {
repeat {
do {
let datagram = try self.tcpClient?.receive()
var byteData = datagram?["data"] as? Data
let dataLength = datagram?["length"] as? Int
let _ = TPCircularBufferProduceBytes(&self.circularBuffer, byteData.bytes, UInt32(decodedLength * 2))
} catch {
fatalError(error.localizedDescription)
}
} while true
}
}
Create Audio Unit

...
var desc = AudioComponentDescription(
componentType: OSType(kAudioUnitType_Output),
componentSubType: OSType(kAudioUnitSubType_VoiceProcessingIO),
componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
componentFlags: 0,
componentFlagsMask: 0
)
let inputComponent = AudioComponentFindNext(nil, &desc)
status = AudioComponentInstanceNew(inputComponent!, &audioUnit)
if status != noErr {
print("Audio component instance new error \(status!)")
}
// Enable IO for playback
status = AudioUnitSetProperty(
audioUnit!,
kAudioOutputUnitProperty_EnableIO,
kAudioUnitScope_Output,
kOutputBus,
&flag,
SizeOf32(flag)
)
if status != noErr {
print("Enable IO for playback error \(status!)")
}
//Use your own format, I have sample rate of 16000 and pcm 16 Bit
var ioFormat = CAStreamBasicDescription(
sampleRate: 16000.0,
numChannels: 1,
pcmf: .int16,
isInterleaved: false
)
//This is playbackCallback 
 var playbackCallback = AURenderCallbackStruct(
inputProc: AudioController_PlaybackCallback, //This is a delegate where audioUnit puts the bytes
inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
)
status = AudioUnitSetProperty(
audioUnit!,
AudioUnitPropertyID(kAudioUnitProperty_SetRenderCallback),
AudioUnitScope(kAudioUnitScope_Input),
kOutputBus,
&playbackCallback,
MemoryLayout<AURenderCallbackStruct>.size.ui
)
if status != noErr {
print("Failed to set recording render callback \(status!)")
}
//Init Audio Unit
status = AudioUnitInitialize(audioUnit!)
if status != noErr {
print("Failed to initialize audio unit \(status!)")
}
//Start AudioUnit
 status = AudioOutputUnitStart(audioUnit!)
if status != noErr {
print("Failed to initialize output unit \(status!)")
}
 
This is my playbackCallback function where I play the audio from Circular Buffer

func performPlayback(
_ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>,
inTimeStamp: UnsafePointer<AudioTimeStamp>,
inBufNumber: UInt32,
inNumberFrames: UInt32,
ioData: UnsafeMutablePointer<AudioBufferList>
) -> OSStatus {
let buffer = ioData[0].mBuffers
let bytesToCopy = ioData[0].mBuffers.mDataByteSize
var bufferTail: UnsafeMutableRawPointer?
var availableBytes: UInt32 = 0
bufferTail = TPCircularBufferTail(&self.circularBuffer, &availableBytes)
let bytesToWrite = min(bytesToCopy, availableBytes)
var bufferList = AudioBufferList(
mNumberBuffers: 1,
mBuffers: ioData[0].mBuffers)
var monoSamples = [Int16]()
let ptr = bufferList.mBuffers.mData?.assumingMemoryBound(to: Int16.self)
monoSamples.append(contentsOf: UnsafeBufferPointer(start: ptr, count: Int(inNumberFrames)))
print(monoSamples)
memcpy(buffer.mData, bufferTail, Int(bytesToWrite))
TPCircularBufferConsume(&self.circularBuffer, bytesToWrite)
return noErr
}
For TPCircularBuffer I used this pod
'TPCircularBuffer', '~> 1.6'

Answer 3

All detail description and sample code is available for
Audiotoolbox / AudioUnit
You can register the Callback to get the PCM data from the AUGraph and send the pcm buffer to the socket.
Some more example of the usage :
https://github.com/rweichler/coreaudio-examples/blob/master/CH08_AUGraphInput/main.cpp