I use AVAudioMixerNode to change audio format. this entry helped me a lot. Below code gives me data i want. But i hear my own voice on phone's speaker. How can i prevent it?
func startAudioEngine()
{
engine = AVAudioEngine()
guard let engine = engine, let input = engine.inputNode else {
// #TODO: error out
return
}
let downMixer = AVAudioMixerNode()
//I think you the engine's I/O nodes are already attached to itself by default, so we attach only the downMixer here:
engine.attach(downMixer)
//You can tap the downMixer to intercept the audio and do something with it:
downMixer.installTap(onBus: 0, bufferSize: 2048, format: downMixer.outputFormat(forBus: 0), block: //originally 1024
{ (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
//i get audio data here
}
)
//let's get the input audio format right as it is
let format = input.inputFormat(forBus: 0)
//I initialize a 16KHz format I need:
let format16KHzMono = AVAudioFormat.init(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 11025.0, channels: 1, interleaved: true)
//connect the nodes inside the engine:
//INPUT NODE --format-> downMixer --16Kformat--> mainMixer
//as you can see I m downsampling the default 44khz we get in the input to the 16Khz I want
engine.connect(input, to: downMixer, format: format)//use default input format
engine.connect(downMixer, to: engine.outputNode, format: format16KHzMono)//use new audio format
engine.prepare()
do {
try engine.start()
} catch {
// #TODO: error out
}
}
You can hear your microphone recording through your speakers because your microphone is connected to downMixer, which is connected to engine.outputNode. You could probably just mute the output for the downMixer if you aren't using it with other inputs:
downMixer.outputVolume = 0.0
I did it like this to change the frequency to 48000Hz / 16 bit per sample / 2 channels, and save it to wave file:
let outputAudioFileFormat = [AVFormatIDKey: Int(kAudioFormatLinearPCM), AVSampleRateKey: 48000, AVNumberOfChannelsKey: 2, AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue]
let audioRecordingFormat : AVAudioFormat = AVAudioFormat.init(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 48000, channels: 2, interleaved: true)!
do{
try file = AVAudioFile(forWriting: url, settings: outputAudioFileFormat, commonFormat: .pcmFormatInt16, interleaved: true)
let recordingSession = AVAudioSession.sharedInstance()
try recordingSession.setPreferredInput(input)
try recordingSession.setPreferredSampleRate(audioRecordingFormat.sampleRate)
engine.inputNode.installTap(onBus: 0, bufferSize: 1024, format: audioRecordingFormat, block: self.bufferAvailable)
engine.connect(engine.inputNode, to: engine.outputNode, format: audioRecordingFormat) //configure graph
}
catch
{
debugPrint("Could not initialize the audio file: \(error)")
}
And the function block
func bufferAvailable(buffer: AVAudioPCMBuffer, time: AVAudioTime)
{
do
{
try self.file?.write(from: buffer)
if self.onBufferAvailable != nil {
DispatchQueue.main.async {
self.onBufferAvailable!(buffer) // outside function used for analyzing and displaying a wave meter
}
}
}
catch{
self.stopEngine()
DispatchQueue.main.async {
self.onRecordEnd(false)
}
}
}
The stopEngine function is this, you should call it also when you want to stop the recording:
private func stopEngine()
{
self.engine.inputNode.removeTap(onBus: 0)
self.engine.stop()
}
Related
I'm trying to tap the microphone and also get SRC.
#main
struct TapSilenceApp: App {
let engine = AVAudioEngine()
let mixer = AVAudioMixerNode()
let mixer2 = AVAudioMixerNode()
init() {
let format = AVAudioFormat(standardFormatWithSampleRate: 16000, channels: 2)!
let inputNode = engine.inputNode
// Try using two mixers:
engine.attach(mixer)
engine.attach(mixer2)
engine.connect(inputNode, to: mixer, format: nil)
engine.connect(mixer, to: mixer2, format: format)
mixer2.installTap(onBus: 0, bufferSize: 256, format: format) { buffer, _ in
assert(!buffer.isSilent)
}
print("engine: \(engine)")
engine.prepare()
try! engine.start()
}
var body: some Scene {
WindowGroup {
ContentView()
}
}
}
That throws an exception with AUGraphParser::InitializeActiveNodesInInputChain(ThisGraph, *GetInputNode()). (AVAudioEngine wins the award for lousiest error messages of any Apple API)
Interestingly, when I remove the call to installTap, it runs.
I still need the tap of course. So trying to connect mixer2 to the output, I do get the tap to run, but the buffers are silent.
Here's the version that's silent:
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(.playAndRecord, options: .defaultToSpeaker)
} catch {
print("Could not set audio category: \(error.localizedDescription)")
}
let format = AVAudioFormat(standardFormatWithSampleRate: 16000, channels: 2)!
let inputNode = engine.inputNode
// Try using two mixers:
engine.attach(mixer)
engine.attach(mixer2)
engine.connect(inputNode, to: mixer, format: nil)
engine.connect(mixer, to: mixer2, format: format)
engine.connect(mixer2, to: engine.mainMixerNode, format: nil)
mixer2.installTap(onBus: 0, bufferSize: 256, format: mixer2.outputFormat(forBus: 0)) { buffer, _ in
// assert(!buffer.isSilent)
print("max value: \(buffer.maxValue)")
}
print("engine: \(engine)")
engine.prepare()
try! engine.start()
If I just put a tap on the inputNode and remove the mixers, I do get audio input, but of course no SRC.
I want to record from iphone microphone and convert to ulaw format streaming data,I guess that is pcm data but I got noise.
What audio format is installTap buff? How can I do to got ulaw data format?
I can got it from AVAudioRecorder but I'm not to got a file.
Do change format settings 'AVFormatIDKey=kAudioFormatULaw' will got crash.
func testMicrophoneRecording1 () throws {
let tapNode: AVAudioNode = mixerNode
let format = tapNode.outputFormat(forBus: 0)
tapNode.installTap(onBus: 0, bufferSize: 1024, format: format, block: {
(buffer, time) in
let d = buffer.toNSData() as Data
let ulaw_data = convert_pcm_(to_ulaw: d)
sendUlawDataToDevice(data: ulaw_data)
})
try engine.start()
}
and connections is:
func makeConnections() {
let inputNode = engine.inputNode
let inputFormat = inputNode.outputFormat(forBus: 0)
engine.connect(inputNode, to: mixerNode, format: inputFormat)
let mainMixerNode = engine.mainMixerNode
let mixerFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 44100, channels: 1, interleaved: true)
engine.connect(mixerNode, to: mainMixerNode, format: mixerFormat)
}
I have got pcm from microphone and convert to ulaw,form this example:
https://github.com/Epskampie/ios-coreaudio-example
I am a beginner in working with sounds and AVAudioEngine in IOS, and I'm developing an application that captures the audio samples as buffers and analyzes it. Furthermore, the sample rate must be 8 kHz with an integer16 PCM data, but when I try to record from the inputNode and convert the data to 8 kHz, it shows 0s in the buffer. However, when I set the commonFormat to .pcmFormatFloat32 it works fine.
My Code:
let inputNode = audioEngine.inputNode
let downMixer = AVAudioMixerNode()
let main = audioEngine.mainMixerNode
let format = inputNode.inputFormat(forBus: 0)
let format16KHzMono = AVAudioFormat(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 8000, channels: 1, interleaved: true)
audioEngine.attach(downMixer)
downMixer.installTap(onBus: 0, bufferSize: 640, format: format16KHzMono) { (buffer, time) -> Void in
do{
print(buffer.description)
if let channel1Buffer = buffer.int16ChannelData?[0] {
// print(channel1Buffer[0])
for i in 0 ... Int(buffer.frameLength-1) {
print((channel1Buffer[i])) //prints 0s :(
}
}
}
}
audioEngine.connect(inputNode, to: downMixer, format: format)
audioEngine.connect(downMixer, to: main, format: format16KHzMono)
audioEngine.prepare()
try! audioEngine.start()
Thanks
I am a beginner in working with sound processing and AVAudioEngine in iOS, and I'm developing an application that captures the audio samples as a buffer and analyzes it. Furthermore, the sample rate must be 8000 kHz and also must be encoded as PCM16Bit, but the default inputNode in the AVAudioEngine is 44.1 kHz.
In Android, the process is quite simple:
AudioRecord audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC,
8000, AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT, bufferSize);
and then start the reading function for the buffer.
I searched a lot, but I didn't find any similar example. Instead, all the examples in which I encountered are capturing the samples in the default node's sample rate(44.1 kHz) like:
let input = audioEngine.inputNode
let inputFormat = input.inputFormat(forBus: 0)
input.installTap(onBus: 0, bufferSize: 640, format: inputFormat) { (buffer, time) -> Void in
print(inputFormat)
if let channel1Buffer = buffer.floatChannelData?[0] {
for i in 0...Int(buffer.frameLength-1) {
print(channel1Buffer[i])
}
}
}
try! audioEngine.start()
So I would like to capture audio samples using AVAudioEngine with 8000 kHz sample rate and PCM16Bit encoding.
Edit:
I reached a solution to transform the input to 8 kHz:
let inputNode = audioEngine.inputNode
let downMixer = AVAudioMixerNode()
let main = audioEngine.mainMixerNode
let format = inputNode.inputFormat(forBus: 0)
let format16KHzMono = AVAudioFormat(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 8000, channels: 1, interleaved: true)
audioEngine.attach(downMixer)
downMixer.installTap(onBus: 0, bufferSize: 640, format: format16KHzMono) { (buffer, time) -> Void in
do{
print(buffer.description)
if let channel1Buffer = buffer.int16ChannelData?[0] {
// print(channel1Buffer[0])
for i in 0 ... Int(buffer.frameLength-1) {
print((channel1Buffer[i]))
}
}
}
}
audioEngine.connect(inputNode, to: downMixer, format: format)
audioEngine.connect(downMixer, to: main, format: format16KHzMono)
audioEngine.prepare()
try! audioEngine.start()
, but when I use .pcmFormatInt16 it doesn't work. However, when I use .pcmFormatFloat32 it works fine!
Have you checked with settings parameter
let format16KHzMono = AVAudioFormat(settings: [AVFormatIDKey: AVAudioCommonFormat.pcmFormatInt16,
AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue,
AVEncoderBitRateKey: 16,
AVNumberOfChannelsKey: 1,
AVSampleRateKey: 8000.0] as [String : AnyObject])
I'm using AVFoundation framework. Whenever the player plays the buffer, my background music gets stopped so I used below code to allow it to continue playing irrespective of the AVFoundation player.
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.mixWithOthers,.allowBluetooth])
try audioSession.setMode(AVAudioSessionModeDefault)
try audioSession.setActive(true)
It does work but the problem is the quality of the background music gets dramatically affected. The music don't have the bass effects anymore whenever the AVPlayer plays the buffer.
I want the background music uninterrupted while using AVPlayer. Is it possible?
update : I added full code if anyone wants to check. Can feel the difference in background itune music as soon as the app is opened or the session is activated when using this code.
class ViewCosdfntroller: UIViewController {
var engine = AVAudioEngine()
let audioSession = AVAudioSession.sharedInstance()
let player = AVAudioPlayerNode()
let mixer = AVAudioMixerNode()
override func viewDidLoad() {
super.viewDidLoad()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.mixWithOthers,.allowBluetooth])
try audioSession.setMode(AVAudioSessionModeDefault)
try audioSession.setActive(true)
} catch {
}
let input = engine.inputNode
let bus = 0
let inputFormat = input.outputFormat(forBus: bus)
let recordingFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 11025.0, channels: 1, interleaved: false)
engine.attach(player)
engine.attach(mixer)
engine.connect(input, to: mixer, format: input.outputFormat(forBus: 0))
engine.connect(player, to: engine.mainMixerNode, format: recordingFormat)
mixer.installTap(onBus: bus, bufferSize: AVAudioFrameCount(inputFormat.sampleRate * 0.4), format: inputFormat, block: { (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
let Converter:AVAudioConverter = AVAudioConverter.init(from: inputFormat, to: recordingFormat!)!
let newbuffer = AVAudioPCMBuffer(pcmFormat: recordingFormat!,frameCapacity: AVAudioFrameCount((recordingFormat?.sampleRate)! * 0.4))
let inputBlock : AVAudioConverterInputBlock = { (inNumPackets, outStatus) -> AVAudioBuffer? in
outStatus.pointee = AVAudioConverterInputStatus.haveData
let audioBuffer : AVAudioBuffer = buffer
return audioBuffer
}
var error : NSError?
Converter.convert(to: newbuffer!, error: &error, withInputFrom: inputBlock)
self.player.scheduleBuffer(newbuffer!)
})
do {
try! engine.start()
player.play()
} catch {
print(error)
}
}
}
Unless this is some weird mixing quirk, the quality change you report may just be that recording categories change the default audio output device to the tiny, tinny receiver (because telephones, don't ask). Override this behaviour by adding .defaultToSpeaker to your setCategory() call:
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.mixWithOthers,.allowBluetooth, .defaultToSpeaker])
I think you need this one:
try audioSession.setCategory(AVAudioSessionCategoryAmbient)
Documentation:
https://developer.apple.com/documentation/avfoundation/avaudiosessioncategoryambient
When you use this category, audio from other apps mixes with your audio