I am making a basic music app for iOS, where pressing notes causes the corresponding sound to play. I am trying to get multiple sounds stored in buffers to play simultaneously with minimal latency. However, I can only get one sound to play at any time.
I initially set up my sounds using multiple AVAudioPlayer objects, assigning a sound to each player. While it did play multiple sounds simultaneously, it didn't seem like it was capable of starting two sounds at the same time (it seemed like it would delay the second sound just slightly after the first sound was started). Furthermore, if I pressed notes at a very fast rate, it seemed like the engine couldn't keep up, and later sounds would start well after I had pressed the later notes.
I am trying to solve this problem, and from the research I have done, it seems like using the AVAudioEngine to play sounds would be the best method, where I can set up the sounds in an array of buffers, and then have them play back from those buffers.
class ViewController: UIViewController
{
// Main Audio Engine and it's corresponding mixer
var audioEngine: AVAudioEngine = AVAudioEngine()
var mainMixer = AVAudioMixerNode()
// One AVAudioPlayerNode per note
var audioFilePlayer: [AVAudioPlayerNode] = Array(repeating: AVAudioPlayerNode(), count: 7)
// Array of filepaths
let noteFilePath: [String] = [
Bundle.main.path(forResource: "note1", ofType: "wav")!,
Bundle.main.path(forResource: "note2", ofType: "wav")!,
Bundle.main.path(forResource: "note3", ofType: "wav")!]
// Array to store the note URLs
var noteFileURL = [URL]()
// One audio file per note
var noteAudioFile = [AVAudioFile]()
// One audio buffer per note
var noteAudioFileBuffer = [AVAudioPCMBuffer]()
override func viewDidLoad()
{
super.viewDidLoad()
do
{
// For each note, read the note URL into an AVAudioFile,
// setup the AVAudioPCMBuffer using data read from the file,
// and read the AVAudioFile into the corresponding buffer
for i in 0...2
{
noteFileURL.append(URL(fileURLWithPath: noteFilePath[i]))
// Read the corresponding url into the audio file
try noteAudioFile.append(AVAudioFile(forReading: noteFileURL[i]))
// Read data from the audio file, and store it in the correct buffer
let noteAudioFormat = noteAudioFile[i].processingFormat
let noteAudioFrameCount = UInt32(noteAudioFile[i].length)
noteAudioFileBuffer.append(AVAudioPCMBuffer(pcmFormat: noteAudioFormat, frameCapacity: noteAudioFrameCount)!)
// Read the audio file into the buffer
try noteAudioFile[i].read(into: noteAudioFileBuffer[i])
}
mainMixer = audioEngine.mainMixerNode
// For each note, attach the corresponding node to the audioEngine, and connect the node to the audioEngine's mixer.
for i in 0...2
{
audioEngine.attach(audioFilePlayer[i])
audioEngine.connect(audioFilePlayer[i], to: mainMixer, fromBus: 0, toBus: i, format: noteAudioFileBuffer[i].format)
}
// Start the audio engine
try audioEngine.start()
// Setup the audio session to play sound in the app, and activate the audio session
try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.soloAmbient)
try AVAudioSession.sharedInstance().setMode(AVAudioSession.Mode.default)
try AVAudioSession.sharedInstance().setActive(true)
}
catch let error
{
print(error.localizedDescription)
}
}
func playSound(senderTag: Int)
{
let sound: Int = senderTag - 1
// Set up the corresponding audio player to play its sound.
audioFilePlayer[sound].scheduleBuffer(noteAudioFileBuffer[sound], at: nil, options: .interrupts, completionHandler: nil)
audioFilePlayer[sound].play()
}
Each sound should be playing without interrupting the other sounds, only interrupting its own sound when the sounds is played again. However, despite setting up multiple buffers and players, and assigning each one to its own Bus on the audioEngine's mixer, playing one sound still stops any other sounds from playing.
Furthermore, while leaving out .interrupts does prevent sounds from stopping other sounds, these sounds won't play until the sound that is currently playing completes. This means that if I play note1, then note2, then note3, note1 will play, while note2 will only play after note1 finishes, and note3 will only play after note2 finishes.
Edit: I was able to get the audioFilePlayer to reset to the beginning again without using interrupt with the following code in the playSound function.
if audioFilePlayer[sound].isPlaying == true
{
audioFilePlayer[sound].stop()
}
audioFilePlayer[sound].scheduleBuffer(noteAudioFileBuffer[sound], at: nil, completionHandler: nil)
audioFilePlayer[sound].play()
This still leaves me with figuring out how to play these sounds simultaneously, since playing another sound will still stop the currently playing sound.
Edit 2: I found the solution to my problem. My answer is below.
It turns out that having the .interrupt option wasn't the issue (in fact, this actually turned out to be the best way to restart the sound that was playing in my experience, as there was no noticeable pause during the restart, unlike the stop() function). The actual problem that was preventing multiple sounds from playing simultaneously was this particular line of code.
// One AVAudioPlayerNode per note
var audioFilePlayer: [AVAudioPlayerNode] = Array(repeating: AVAudioPlayerNode(), count: 7)
What happened here was that each item of the array was being assigned the exact same AVAudioPlayerNode value, so they were all effectively sharing the same AVAudioPlayerNode. As a result, the AVAudioPlayerNode functions were affecting all of the items in the array, instead of just the specified item. To fix this and give each item a different AVAudioPlayerNode value, I ended up changing the above line so that it starts as an empty array of type AVAudioPlayerNode instead.
// One AVAudioPlayerNode per note
var audioFilePlayer = [AVAudioPlayerNode]()
I then added a new line to append to this array a new AVAudioPlayerNode at the beginning inside of the second for-loop of the viewDidLoad() function.
// For each note, attach the corresponding node to the audioEngine, and connect the node to the audioEngine's mixer.
for i in 0...6
{
audioFilePlayer.append(AVAudioPlayerNode())
// audioEngine code
}
This gave each item in the array a different AVAudioPlayerNode value. Playing a sound or restarting a sound no longer interrupts the other sounds that are currently being played. I can now play any of the notes simultaneously and without any noticeable latency between note press and playback.
I'm having a problem with recording audio from the microphone of my test device to a .caf file in Swift, XCode 9.4.1 using the latest version of AudioKit. In a simple test whereby I send the audio straight from the microphone to the output via an AKBooster, it works just fine and I can hear the mic input coming out of the speakers. I'm more or less following this example, although again using a booster node instead of an oscillator.
The following is my code:
class MicrophoneHandler
{
var microphone : AKMicrophone!
var booster : AKBooster!
var mixer : AKMixer!
var recorder : AKNodeRecorder!
var file : AKAudioFile!
var player : AKAudioPlayer!
init()
{
setupMicrophone()
microphone = AKMicrophone()
booster = AKBooster(microphone) // Stereo amplifier for microphone
mixer = AKMixer(booster)
file = try! AKAudioFile() // File to store recorder output
player = try? AKAudioPlayer(file: file) // Player to play back recorded audio file
//player.looping = true
recorder = try? AKNodeRecorder(node: mixer, file: file)
try? recorder.record()
sleep(5)
let dur = String(format: "%0.3f seconds", recorder.recordedDuration)
print("Stopped. (\(dur) recorded)")
recorder.stop()
//file.exportAsynchronously(name: "Test", baseDir: .documents, exportFormat: .caf){ [weak self] _, _ in
//}
//player.play()
//AudioKit.output = player!
//try? AudioKit.start()
}
func setupMicrophone()
{
// Function to initialise microphone settings
// Adapted from AudioKit example code found here:
// https://audiokit.io/examples/MicrophoneAnalysis
AKSettings.bufferLength = .medium
AKSettings.ioBufferDuration = 0.002 // TODO experiment with this to control latency
do
{
try AKSettings.setSession(category: .playAndRecord, with: .allowBluetoothA2DP) // Set session type & allow streaming to Bluetooth devices
} catch
{
AKLog("Could not set session category.")
}
AKSettings.defaultToSpeaker = true // Output to speaker when audio input is enabled
}
}
I have commented out the export code as the problem doesn't appear to be here. The console displays the following:
AKMicrophone.swift:init():45:Mixer inputs 8
AKAudioPlayer.swift:updatePCMBuffer():533:AKAudioPlayer Warning: "BF848EC0-94F8-4E39-A211-784B001CED72.caf" is an empty file
2018-11-16 17:49:16.936169+0000 VoxBox[2258:6984570] Audio files cannot be non-interleaved. Ignoring setting AVLinearPCMIsNonInterleaved YES.
AKNodeRecorder.swift:record():104:AKNodeRecorder: recording
Stopped. (0.000 seconds recorded)
As you can see, the recorder appears not to be recording to file for some reason. To my mind, my code should
Initialise the microphone (including settings)
Route the microphone input through a booster followed by a mixer (mixing with an FX bank will happen later)
Create an empty .caf audio file to be written to
Set up a player to play this file when the time comes
Set up a recorder to record the output of the mixer node to the audio file
Record 5 seconds of microphone input to the audio file
Yet for some reason nothing is being recorded. Clearly I am missing something or have misunderstood how the AKNodeRecorder works in this regard. I have read as many StackOverflow questions on similar topics as I can, had a dig through the AudioKit documentation and read a couple of examples from the AudioKit site, but nothing seems to address my particular problem.
Any help would be much appreciated.
Working on a project in swift, where I'm trying to initiate the AVPlayer, and for some reason it trows me an exception saying
AudioHardware.cpp:1200:AudioObjectRemovePropertyListener: AudioObjectRemovePropertyListener: no object with given ID 0.
I guess the issue is with my URL. Here is my code
func initPlayer() {
let url:NSURL = NSURL(string:"https://purelight1-163000.appspot.com/api/user/v2/media/track/60/sample")!
self.playerItem = AVPlayerItem(url: url as URL)
self.player=AVPlayer(playerItem: self.playerItem!)
let playerLayer=AVPlayerLayer(player: self.player!)
playerLayer.frame = CGRect(x: 0, y: 0, width: 10, height: 50) // actually this player layer is not visible
self.view.layer.addSublayer(playerLayer)
}
For me it was because of the Simulator's audio output.
I was using an audio capture app iShowU for different project with headphones setup. But to disable the simulator's audio only I set that "Audio Device Output" to the iShowU setup with the headphones (which weren't plugged in). It finished playing the video that was loaded and then that problem appeared.
Switching the audio output from Simulator > I/O > Audio Output to the System one and rerunning my build resolved my issue.
Xcode is overly chatty. You just need edit your Scheme (Product > Scheme > Edit Scheme) and create a new Environment Variable OS_ACTIVITY_MODE and set its value to disable.
I am having trouble getting positional audio to work in SceneKit. Starting with the SceneKit game template generated by Xcode I have added the following code to the end of the handleTap method:
let ship = scnView.scene!.rootNode.childNode(withName: "ship", recursively: true)!
if let source = SCNAudioSource(fileNamed: "art.scnassets/monoAudioTest.wav")
{
source.volume = 1
source.isPositional = true
source.shouldStream = true
source.loops = true
source.load()
let player = SCNAudioPlayer(source: source)
ship.addAudioPlayer(player)
}
ship.runAction(SCNAction.move(to: SCNVector3(0, 0, -10000), duration: 8))
The audio plays but the volume doesn't decrease as the jet moves away from the camera. Am I missing some steps or making some wrong assumptions?
Cross-posted to the Apple Developer Forums.
As mentioned by Jed Soane, and confirmed by Apple in a Radar, the issue was my audio file was stereo instead of mono. Only mono audio files will work for positional audio.
You should be able to get positional audio with source.shouldStream = false.
I'm developing a music instrument in iOS with two audio samples (high and low pitches) that are played with view touches. The first sample is very short (a half second) and the other is a little bigger (two seconds). When I play repeatedly and fast the low pitch sound, there is an audio click/pop. There is no problem playing the high pitch sound.
Both audio samples have fade in and fade out in their init/end and there is no clip problem with them.
I'm using this code to load the audio files (simplified here):
engine = AVAudioEngine()
mixer = engine.mainMixerNode
let player = AVAudioPlayerNode()
do {
let audioFile = try AVAudioFile(forReading: instrumentURL)
let audioFormat = audioFile.processingFormat
let audioFrameCount = UInt32(audioFile.length)
let audioFileBuffer = AVAudioPCMBuffer(PCMFormat: audioFormat, frameCapacity: audioFrameCount)
try audioFile.readIntoBuffer(audioFileBuffer)
engine.attachNode(player)
engine.connect(player, to: mixer, format: audioFileBuffer.format)
} catch {
print("Init Error!")
}
and this code to play the samples:
player.play()
player.scheduleBuffer(audioFileBuffer, atTime: nil, options: option, completionHandler: nil)
I'm using a similar functionality in Android with the same audio samples without any click/pop problem.
Is this click/pop problem an implementation error?
How can I fix this problem?
Update 1
I just tried another approach, with AVAudioPlayer and I got the same pop/click problem.
Update 2
I think the problem is to start the audio file again before its end. The sound stops abruptly.