I am trying to use AVAudioEngine for listening to mic samples and playing them simultaneously via external speakers or headphones (assuming they are attached to iOS device). I tried the following using AVAudioPlayerNode and it works, but there is too much delay in the audio playback. Is there a way to hear sound realtime without delay?
var engine: AVAudioEngine!
var playerNode: AVAudioPlayerNode!
var mixer: AVAudioMixerNode!
var audioEngineRunning = false
public func setupAudioEngine() {
self.engine = AVAudioEngine()
let input = engine.inputNode
let format = input.inputFormat(forBus: 0)
playerNode = AVAudioPlayerNode()
engine.attach(playerNode)
self.mixer = engine.mainMixerNode
engine.connect(self.playerNode, to: self.mixer, format: playerNode.outputFormat(forBus: 0))
engine.inputNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: { buffer, time in
self.playerNode.scheduleBuffer(buffer, completionHandler: nil)
})
do {
engine.prepare()
try self.engine.start()
audioEngineRunning = true
self.playerNode.play()
}
catch {
print("error couldn't start engine")
audioEngineRunning = false
}
}
Related
currently trying to get my AKSampler to play sounds that I send it but not having much luck getting audio to output. My AKMidiCallbackInstrument is properly logging the notes playing (although I'm seeing the print for each note twice..) However, the call to my sampler is not producing any audio and I can't figure out why.
class Sequencer {
var sampler: AKSampler
var sequencer: AKAppleSequencer
var mixer: AKMixer
init() {
sampler = AKSampler()
sequencer = AKAppleSequencer()
mixer=AKMixer(sampler)
let midicallback = AKMIDICallbackInstrument()
let url = Bundle.main.url(forResource: "UprightPianoKW-20190703", withExtension: "sfz")!;
let track = sequencer.newTrack()
track?.setMIDIOutput(midicallback.midiIn)
sampler.loadSFZ(url: url)
//generate some notes and add thtem to the track
generateSequence()
midicallback >>> mixer
AudioKit.output = mixer
AKSettings.playbackWhileMuted = true
AKSettings.audioInputEnabled = true
midicallback.callback = { status, note, vel in
guard let status = AKMIDIStatus(byte: status),
let type = status.type,
type == .noteOn else { return print("note off: \(note)") }
print("note on: \(note)")
self.sampler.play(noteNumber: note, velocity: vel) }
}
func play() {
try? AudioKit.start()
sequencer.rewind()
sequencer.play()
try? AudioKit.stop()
}
func stop() {
sequencer.stop()
}
you need to connect your sampler to the mixer:
sampler >>> mixer
Fwiw,
midicallback >>> mixer isn't necessary with AKAppleSequencer/AKMIDICallbackInstrument although it would be with AKSequencer/AKCallbackInstrument
I was following this question, but the tone that I try to play with an AVAudioPCMBuffer does not play. The code is pretty simple:
class Player: NSObject {
var engine = AVAudioEngine()
var player = AVAudioPlayerNode()
var mixer: AVAudioMixerNode!
var buffer: AVAudioPCMBuffer!
override init() {
mixer = engine.mainMixerNode
buffer = AVAudioPCMBuffer(pcmFormat: player.outputFormat(forBus: 0), frameCapacity: 100)
buffer.frameLength = 100
let sr = mixer.outputFormat(forBus: 0).sampleRate
let nChannels = mixer.outputFormat(forBus: 0).channelCount
var i = 0
while i < Int(buffer.frameLength) {
let val = sin(441 * Double(i) * Double.pi / sr)
buffer.floatChannelData?.pointee[i] = Float(val * 0.5)
i += Int(nChannels)
}
engine.attach(player)
engine.connect(player, to: mixer, format: player.outputFormat(forBus: 0))
engine.prepare()
}
func play() {
do {
try engine.start()
} catch {
print(error)
}
player.scheduleBuffer(buffer, at: nil, options: .loops) {
print("Played!")
}
player.play()
}
}
For some reason, though, the iPhone does not make any sound. In my ViewController, I have this:
class ViewController: UIViewController {
var player = Player()
override func viewDidAppear(_ animated: Bool) {
player.play()
}
}
As you can see, player is a class variable, so it should not be deallocated from memory.
When I run the app on my physical device (iPhone 6s iOS 11), it does not work, yet it does work on the simulator. Why is this not making any sound, and how can I fix it?
Thanks in advance!
Make sure your device is not on silent mode.
I just created a project that you can download for testing by yourself: https://github.com/mugx/TestSound
I am doing research over four days, But I am not found any solution for calling over Bluetooth between two iOS devices within a distance.
I found that audio streaming is possible between two iOS devices using multipeer connectivity framework but this is not helpful for me. I want real time voice chat between two devices over Bluetooth.
Is there any CO-DAC for voice over Bluetooth?
My code is:
var engine = AVAudioEngine()
var file: AVAudioFile?
var player = AVAudioPlayerNode()
var input:AVAudioInputNode?
var mixer:AVAudioMixerNode?
override func viewDidLoad() {
super.viewDidLoad()
mixer = engine.mainMixerNode
input = engine.inputNode
engine.connect(input!, to: mixer!, format: input!.inputFormat(forBus: 0))
}
#IBAction func btnStremeDidClicked(_ sender: UIButton) {
mixer?.installTap(onBus: 0, bufferSize: 2048, format: mixer?.outputFormat(forBus: 0), block: { (buffer: AVAudioPCMBuffer, AVAudioTime) in
let byteWritten = self.audioBufferToData(audioBuffer: buffer).withUnsafeBytes {
self.appDelegate.mcManager.outputStream?.write($0, maxLength: self.audioBufferToData(audioBuffer: buffer).count)
}
print(byteWritten ?? 0)
print("Write")
})
do {
try engine.start()
}catch {
print(error.localizedDescription)
}
}
func audioBufferToData(audioBuffer: AVAudioPCMBuffer) -> Data {
let channelCount = 1
let bufferLength = (audioBuffer.frameCapacity * audioBuffer.format.streamDescription.pointee.mBytesPerFrame)
let channels = UnsafeBufferPointer(start: audioBuffer.floatChannelData, count: channelCount)
let data = Data(bytes: channels[0], count: Int(bufferLength))
return data
}
Thanks in Advance :)
Why is MultipeerConnectivity not helpful for you? It is a great way to stream audio over bluetooth or even wifi.
When you call this:
audioEngine.installTap(onBus: 0, bufferSize: 17640, format: localInputFormat) {
(buffer, when) -> Void in
You need to use the buffer, which has type AVAudioPCMBuffer. You then need to convert that to NSData and write to the outputStream that you would've opened with the peer:
data = someConverstionMethod(buffer)
_ = stream!.write(data.bytes.assumingMemoryBound(to: UInt8.self), maxLength: data.length)
Then on the other device you need to read from the stream and convert from NSData back to an AVAudioPCMBuffer, and then you can use an AVAudioPlayer to playback the buffer.
I have done this before with a very minimal delay.
I’m trying to get some audio to be able to have the pitch adjusted whilst playing. I’m very new to Swift and iOS, but my initial attempt was to just change timePitchNode.pitch whilst it was playing; however, it wouldn’t update whilst playing. My current attempt is to reset audioEngine, and have it just resume from where it was playing (below). How do I determine where the audio currently is, and how do I get it to resume from there?
var audioFile: AVAudioFile?
var audioEngine: AVAudioEngine?
var audioPlayerNode: AVAudioPlayerNode?
var pitch: Int = 1 {
didSet {
playResumeAudio()
}
}
…
func playResumeAudio() {
var currentTime: AVAudioTime? = nil
if audioPlayerNode != nil {
let nodeTime = audioPlayerNode!.lastRenderTime!
currentTime = audioPlayerNode!.playerTimeForNodeTime(nodeTime)
}
if audioEngine != nil {
audioEngine!.stop()
audioEngine!.reset()
}
audioEngine = AVAudioEngine()
audioPlayerNode = AVAudioPlayerNode()
audioEngine!.attachNode(audioPlayerNode!)
let timePitchNode = AVAudioUnitTimePitch()
timePitchNode.pitch = Float(pitch * 100)
timePitchNode.rate = rate
audioEngine!.attachNode(timePitchNode)
audioEngine!.connect(audioPlayerNode!, to: timePitchNode, format: nil)
audioEngine!.connect(timePitchNode, to: audioEngine!.outputNode, format: nil)
audioPlayerNode!.scheduleFile(audioFile!, atTime: nil, completionHandler: nil)
let _ = try? audioEngine?.start()
audioPlayerNode!.playAtTime(currentTime)
}
I was being dumb apparently. You can modify the pitch during playback, and it does update. No need to reset any audio, just mutate the node as it’s playing, and it’ll work.
Background: I found one of Apple WWDC sessions called "AVAudioEngine in Practice" and am trying to make something similar to the last demo shown at 43:35 (https://youtu.be/FlMaxen2eyw?t=2614). I'm using SpriteKit instead of SceneKit but the principle is the same: I want to generate spheres, throw them around and when they collide the engine plays a sound, unique to each sphere.
Problems:
I want a unique AudioPlayerNode attached to each SpriteKitNode so that I can play a different sound for each sphere. i.e Right now, if I create two spheres and set a different pitch for each of their AudioPlayerNode, only the most recently created AudioPlayerNode seems to be playing, even when the original sphere collides. During the demo, he mentions "I'm tying a player, a dedicated player to each ball". How would I go about doing that?
There are audio clicks/artefacts every time a new collision happens. I'm assuming this has to do with the AVAudioPlayerNodeBufferOptions and/or the fact that I'm trying to create, schedule and consume buffers very quickly each time contact occurs, which is not the most efficient method. What would be a good work around for this?
Code: As mentioned in the video, "...for every ball that's born into this world, a new player node is also created". I have a separate class for the spheres, with a method that returns a SpriteKitNode and also creates an AudioPlayerNode every time it is called :
class Sphere {
var sphere: SKSpriteNode = SKSpriteNode(color: UIColor(), size: CGSize())
var sphereScale: CGFloat = CGFloat(0.01)
var spherePlayer = AVAudioPlayerNode()
let audio = Audio()
let sphereCollision: UInt32 = 0x1 << 0
func createSphere(position: CGPoint, pitch: Float) -> SKSpriteNode {
let texture = SKTexture(imageNamed: "Slice")
let collisionTexture = SKTexture(imageNamed: "Collision")
// Define the node
sphere = SKSpriteNode(texture: texture, size: texture.size())
sphere.position = position
sphere.name = "sphere"
sphere.physicsBody = SKPhysicsBody(texture: collisionTexture, size: sphere.size)
sphere.physicsBody?.dynamic = true
sphere.physicsBody?.mass = 0
sphere.physicsBody?.restitution = 0.5
sphere.physicsBody?.usesPreciseCollisionDetection = true
sphere.physicsBody?.categoryBitMask = sphereCollision
sphere.physicsBody?.contactTestBitMask = sphereCollision
sphere.zPosition = 1
// Create AudioPlayerNode
spherePlayer = audio.createPlayer(pitch)
return sphere
}
Here's my Audio Class with which I create AudioPCMBuffers and AudioPlayerNodes
class Audio {
let engine: AVAudioEngine = AVAudioEngine()
func createBuffer(name: String, type: String) -> AVAudioPCMBuffer {
let audioFilePath = NSBundle.mainBundle().URLForResource(name as String, withExtension: type as String)!
let audioFile = try! AVAudioFile(forReading: audioFilePath)
let buffer = AVAudioPCMBuffer(PCMFormat: audioFile.processingFormat, frameCapacity: UInt32(audioFile.length))
try! audioFile.readIntoBuffer(buffer)
return buffer
}
func createPlayer(pitch: Float) -> AVAudioPlayerNode {
let player = AVAudioPlayerNode()
let buffer = self.createBuffer("PianoC1", type: "wav")
let pitcher = AVAudioUnitTimePitch()
let delay = AVAudioUnitDelay()
pitcher.pitch = pitch
delay.delayTime = 0.2
delay.feedback = 90
delay.wetDryMix = 0
engine.attachNode(pitcher)
engine.attachNode(player)
engine.attachNode(delay)
engine.connect(player, to: pitcher, format: buffer.format)
engine.connect(pitcher, to: delay, format: buffer.format)
engine.connect(delay, to: engine.mainMixerNode, format: buffer.format)
engine.prepare()
try! engine.start()
return player
}
}
In my GameScene class I then test for collision, schedule a buffer and play the AudioPlayerNode if contact has occurred
func didBeginContact(contact: SKPhysicsContact) {
let firstBody: SKPhysicsBody = contact.bodyA
if (firstBody.categoryBitMask & sphere.sphereCollision != 0) {
let buffer1 = audio.createBuffer("PianoC1", type: "wav")
sphere.spherePlayer.scheduleBuffer(buffer1, atTime: nil, options: AVAudioPlayerNodeBufferOptions.Interrupts, completionHandler: nil)
sphere.spherePlayer.play()
}
}
I'm new to Swift and only have basic knowledge of programming so any suggestion/criticism is welcome.
I've been working on AVAudioEngine in scenekit and trying to do something else, but this will be what you are looking for:
https://developer.apple.com/library/mac/samplecode/AVAEGamingExample/Listings/AVAEGamingExample_AudioEngine_m.html
It explains the process of:
1-Instantiating your own AVAudioEngine sub-class
2-Methods to load PCMBuffers for each AVAudioPlayer
3-Changing your Environment node's parameters to accomodate the reverb for the large number of pinball objects
Edit: Converted, tested and added a few features:
1-You create a subclass of AVAudioEngine, name it AudioLayerEngine for example. This is to access the AVAudioUnit effects such as distortion, delay, pitch and many of the other effects available as AudioUnits.
2-Initialise by setting up some configurations for the audio engine, such as rendering algorithm, exposing the AVAudioEnvironmentNode to play with 3D positions of your SCNNode objects or SKNode objects if you are in 2D but want 3D effects
3-Create some helper methods to load presets for each AudioUnit effect you want
4-Create a helper method to create an audio player then add it to whatever node you want, as many times as you want since that SCNNode accepts a .audioPlayers methods which returns [AVAudioPlayer] or [SCNAudioPlayer]
5-Start playing.
I've pasted the entire class for reference so that you can then structure it as you wish, but keep in mind that if you are coupling this with SceneKit or SpriteKit, you use this audioEngine to manage all your sounds instead of SceneKit's internal AVAudioEngine. This means that you instantiate this in your gameView during the AwakeFromNib method
import Foundation
import SceneKit
import AVFoundation
class AudioLayerEngine:AVAudioEngine{
var engine:AVAudioEngine!
var environment:AVAudioEnvironmentNode!
var outputBuffer:AVAudioPCMBuffer!
var voicePlayer:AVAudioPlayerNode!
var multiChannelEnabled:Bool!
//audio effects
let delay = AVAudioUnitDelay()
let distortion = AVAudioUnitDistortion()
let reverb = AVAudioUnitReverb()
override init(){
super.init()
engine = AVAudioEngine()
environment = AVAudioEnvironmentNode()
engine.attachNode(self.environment)
voicePlayer = AVAudioPlayerNode()
engine.attachNode(voicePlayer)
voicePlayer.volume = 1.0
outputBuffer = loadVoice()
wireEngine()
startEngine()
voicePlayer.scheduleBuffer(self.outputBuffer, completionHandler: nil)
voicePlayer.play()
}
func startEngine(){
do{
try engine.start()
}catch{
print("error loading engine")
}
}
func loadVoice()->AVAudioPCMBuffer{
let URL = NSURL(fileURLWithPath: NSBundle.mainBundle().pathForResource("art.scnassets/sounds/interface/test", ofType: "aiff")!)
do{
let soundFile = try AVAudioFile(forReading: URL, commonFormat: AVAudioCommonFormat.PCMFormatFloat32, interleaved: false)
outputBuffer = AVAudioPCMBuffer(PCMFormat: soundFile.processingFormat, frameCapacity: AVAudioFrameCount(soundFile.length))
do{
try soundFile.readIntoBuffer(outputBuffer)
}catch{
print("somethign went wrong with loading the buffer into the sound fiel")
}
print("returning buffer")
return outputBuffer
}catch{
}
return outputBuffer
}
func wireEngine(){
loadDistortionPreset(AVAudioUnitDistortionPreset.MultiCellphoneConcert)
engine.attachNode(distortion)
engine.attachNode(delay)
engine.connect(voicePlayer, to: distortion, format: self.outputBuffer.format)
engine.connect(distortion, to: delay, format: self.outputBuffer.format)
engine.connect(delay, to: environment, format: self.outputBuffer.format)
engine.connect(environment, to: engine.outputNode, format: constructOutputFormatForEnvironment())
}
func constructOutputFormatForEnvironment()->AVAudioFormat{
let outputChannelCount = self.engine.outputNode.outputFormatForBus(1).channelCount
let hardwareSampleRate = self.engine.outputNode.outputFormatForBus(1).sampleRate
let environmentOutputConnectionFormat = AVAudioFormat(standardFormatWithSampleRate: hardwareSampleRate, channels: outputChannelCount)
multiChannelEnabled = false
return environmentOutputConnectionFormat
}
func loadDistortionPreset(preset: AVAudioUnitDistortionPreset){
distortion.loadFactoryPreset(preset)
}
func createPlayer(node: SCNNode){
let player = AVAudioPlayerNode()
distortion.loadFactoryPreset(AVAudioUnitDistortionPreset.SpeechCosmicInterference)
engine.attachNode(player)
engine.attachNode(distortion)
engine.connect(player, to: distortion, format: outputBuffer.format)
engine.connect(distortion, to: environment, format: constructOutputFormatForEnvironment())
let algo = AVAudio3DMixingRenderingAlgorithm.HRTF
player.renderingAlgorithm = algo
player.reverbBlend = 0.3
player.renderingAlgorithm = AVAudio3DMixingRenderingAlgorithm.HRTF
}
}