AVAudioPlayerNode playAt() not synchronized - ios

I have one main AVAudioPlayerNode that plays constantly. And there are other players that are put in the queue and play when the main player reaches a certain point.
These AVAudioPlayerNode`s should be synchronized to a millisecond. Sometimes 4-10 pieces can be started at a time.
How its works - I store lastRenderTime of the main player on point when need start all scheduled players and then start all needed player with
player.start(at: lastRenderTime)
Usually, it's working well and without any latency between sounds.
But, I got some latency on this case:
Running on OLD devices(iPad 2, iPhone 5 etc)
ONLY when MANY players the first time after application startup (For example on iPhone 7 I run 8 players in same time after the application is running and I hear desynchronization between sounds), but all players are running with same player.start(at: lastRenderTime) time.
All player actions I run async with custom concurrent queue.
Spend 3 days on trying fix this. I would like for any pieces of advice.
Some code:
Here is how I starting main player -
self.scheduleBuffer(audioFileBuffer, at: nil, options: [], completionHandler: {
lastRenderTime = lastRenderTime + audioBufferLength //Schematic showing logic that after each loop I calculate current render time value.
if self.isStopPressed == false { //Run fileEndCompletion only when player seek to end.
fileEndCompletionHandler() //here is callback that running all scheduled players.
if loop { // Recursive play this file again.
self.playAudioBuffer(audioFileBuffer: audioFileBuffer, loop: loop, fileEndCompletionHandler: fileEndCompletionHandler)
}
}
})
Example of code that running all scheduled player.
AudioEnginePlayer.sharedInstance.audioQueue.async {
if !self.isPlaying {
self.volume = 1.0
self.audioFile.framePosition = frameCount.startingFrame
self.prepare(withFrameCount: frameCount.frameCount)
self.play(at: AudioEnginePlayer.sharedInstance.loopMsLastStartRenderTime) //Calculated lastRenderTime
}
self.scheduleSegment(self.audioFile, startingFrame: frameCount.startingFrame, frameCount: frameCount.frameCount, at: nil) {
//WARNING: COMPLETION HANDLER RUNNING EARLY THEN FILE IS COMPLETED
if self.isStopPressed == false { //Run fileEndCompletion only when player seek to end.
fileEndCompletionHandler()
if loop {
self.playAudioFile(loop: loop, startTime: 0, fileEndCompletionHandler: fileEndCompletionHandler)
}
}
}
}

Related

avplayer pause or stop notification based on the rate

I need to send notifications when the AVPlayer is Play/Paused and Stopped.
For play and pause below is the code
if (self.player.rate > 0.0f) {
NSLog(#" Playing ..")
}
if (self.player.rate == 0.0f) {
NSLog(#" Paused ..")
}
But for stopped also the rate = 0.0 then is there any other property or way to identify the difference between paused and stopped.
For both, paused and stopped the rate = 0.0 and hence need another way for it.
Thanks
There is no stop command for an AVPlayer. So there is no such thing as stopped as distinct from paused. Either the rate is zero (not playing) or it is more than zero (playing).
You can distinguish where the player is within its item (currentTime) so you can tell whether we are at the start, end, or middle; and you can arrange to be notified periodically during play or when the end is reached.
Apart from that, there are no distinctions to be drawn.

Audio won't play after app interrupted by phone call iOS

I have a problem in my SpriteKit game where audio using playSoundFileNamed(_ soundFile:, waitForCompletion:) will not play after the app is interrupted by a phone call. (I also use SKAudioNodes in my app which aren't affected but I really really really want to be able to use the SKAction playSoundFileNamed as well.)
Here's the gameScene.swift file from a stripped down SpriteKit game template which reproduces the problem. You just need to add an audio file to the project and call it "note"
I've attached the code that should reside in appDelegate to a toggle on/off button to simulate the phone call interruption. That code 1) Stops AudioEngine then deactivates AVAudioSession - (normally in applicationWillResignActive) ... and 2) Activates AVAudioSession then Starts AudioEngine - (normally in applicationDidBecomeActive)
The error:
AVAudioSession.mm:1079:-[AVAudioSession setActive:withOptions:error:]: Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.
This occurs when attempting to deactivate the audio session but only after a sound has been played at least once.
to reproduce:
1) Run the app
2) toggle the engine off and on a few times. No error will occur.
3) Tap the playSoundFileNamed button 1 or more times to play the sound.
4) Wait for sound to stop
5) Wait some more to be sure
6) Tap Toggle Audio Engine button to stop the audioEngine and deactivate session -
the error occurs.
7) Toggle the engine on and of a few times to see session activated, session deactivated, session activated printed in debug area - i.e. no errors reported.
8) Now with session active and engine running, playSoundFileNamed button will not play the sound anymore.
What am I doing wrong?
import SpriteKit
import AVFoundation
class GameScene: SKScene {
var toggleAudioButton: SKLabelNode?
var playSoundFileButton: SKLabelNode?
var engineIsRunning = true
override func didMove(to view: SKView) {
toggleAudioButton = SKLabelNode(text: "toggle Audio Engine")
toggleAudioButton?.position = CGPoint(x:20, y:100)
toggleAudioButton?.name = "toggleAudioEngine"
toggleAudioButton?.fontSize = 80
addChild(toggleAudioButton!)
playSoundFileButton = SKLabelNode(text: "playSoundFileNamed")
playSoundFileButton?.position = CGPoint(x: (toggleAudioButton?.frame.midX)!, y: (toggleAudioButton?.frame.midY)!-240)
playSoundFileButton?.name = "playSoundFileNamed"
playSoundFileButton?.fontSize = 80
addChild(playSoundFileButton!)
}
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
if let touch = touches.first {
let location = touch.location(in: self)
let nodes = self.nodes(at: location)
for spriteNode in nodes {
if spriteNode.name == "toggleAudioEngine" {
if engineIsRunning { // 1 stop engine, 2 deactivate session
scene?.audioEngine.stop() // 1
toggleAudioButton!.text = "engine is paused"
engineIsRunning = !engineIsRunning
do{
// this is the line that fails when hit anytime after the playSoundFileButton has played a sound
try AVAudioSession.sharedInstance().setActive(false) // 2
print("session deactivated")
}
catch{
print("DEACTIVATE SESSION FAILED")
}
}
else { // 1 activate session/ 2 start engine
do{
try AVAudioSession.sharedInstance().setActive(true) // 1
print("session activated")
}
catch{
print("couldn't setActive = true")
}
do {
try scene?.audioEngine.start() // 2
toggleAudioButton!.text = "engine is running"
engineIsRunning = !engineIsRunning
}
catch {
//
}
}
}
if spriteNode.name == "playSoundFileNamed" {
self.run(SKAction.playSoundFileNamed("note", waitForCompletion: false))
}
}
}
}
}
Let me save you some time here: playSoundFileNamed sounds wonderful in theory, so wonderful that you might say use it in an app you spent 4 years developing until one day you realize it’s not just totally broken on interruptions but will even crash your app in the most critical of interruptions, your IAP. Don’t do it. I’m still not entirely sure whether SKAudioNode or AVPlayer is the answer, but it may depend on your use case. Just don’t do it.
If you need scientific evidence, create an app and create a for loop that playSoundFileNamed whatever you want in touchesBegan, and see what happens to your memory usage. The method is a leaky piece of garbage.
EDITED FOR OUR FINAL SOLUTION:
We found having a proper number of preloaded instances of AVAudioPlayer in memory with prepareToPlay() was the best method. The SwiftySound audio class uses an on-the-fly generator, but making AVAudioPlayers on the fly created slowdown in animation. We found having a max number of AVAudioPlayers and checking an array for those where isPlaying == false was simplest and best; if one isn't available you don't get sound, similar to what you likely saw with PSFN if you had it playing lots of sounds on top of each other. Overall, we have not found an ideal solution, but this was close for us.
In response to Mike Pandolfini’s advice not to use playSoundFileNamed I’ve converted my code to only use SKAudioNodes.
(and sent the bug report to apple).
I then found that some of these SKAudioNodes don’t play after app interruption either … and I’ve stumbled across a fix.
You need to tell each SKAudioNode to stop() as the app resigns to, or returns from the background - even if they’re not playing.
(I'm now not using any of the code in my first post which stops the audio engine and deactivates the session)
The problem then became how to play the same sound rapidly where it possibly plays over itself. That was what was so good about playSoundFileNamed.
1) The SKAudioNode fix:
Preload your SKAudioNodes i.e.
let sound = SKAudioNode(fileNamed: "super-20")
In didMoveToView add them
sound.autoplayLooped = false
addChild(sound)
Add a willResignActive notification
notificationCenter.addObserver(self, selector:#selector(willResignActive), name:UIApplication.willResignActiveNotification, object: nil)
Then create the selector’s function which stops all audioNodes playing:
#objc func willResignActive() {
for node in self.children {
if NSStringFromClass(type(of: node)) == “SKAudioNode" {
node.run(SKAction.stop())
}
}
}
All SKAudioNodes now play reliably after app interrupt.
2) To replicate playSoundFileNamed’s ability to play the short rapid repeating sounds or longer sounds that may need to play more than once and therefore could overlap, create/preload more than 1 property for each sound and use them like this:
let sound1 = SKAudioNode(fileNamed: "super-20")
let sound2 = SKAudioNode(fileNamed: "super-20")
let sound3 = SKAudioNode(fileNamed: "super-20")
let sound4 = SKAudioNode(fileNamed: "super-20")
var soundArray: [SKAudioNode] = []
var soundCounter: Int = 0
in didMoveToView
soundArray = [sound1, sound2, sound3, sound4]
for sound in soundArray {
sound.autoplayLooped = false
addChild(sound)
}
Create a play function
func playFastSound(from array:[SKAudioNode], with counter:inout Int) {
counter += 1
if counter > array.count-1 {
counter = 0
}
array[counter].run(SKAction.play())
}
To play a sound pass that particular sound's array and its counter to the play function.
playFastSound(from: soundArray, with: &soundCounter)

How to get animation to work at exact points during playback of a music file?

Question:
In Swift code, apart from using an NSTimer, how can I get animations
to start at exact points during playback of a music file played using AVFoundation?
Background
I have a method that plays a music file using AVFoundation (below). I also have UIView animations that I want to start at exact points during the music file being played.
One way I could achieve this is using an NSTimer, but that has the potential to get out of sync or not be exact enough.
Is there a method that I can tap into AVFoundation accessing the music file's time elapsed (time counter), so when certain points during the music playback arrive, animations start?
Is there an event / notification that AVFoundation triggers that gives a constant stream of time elapsed since the music file has started playing?
For example
At 0:52.50 (52 seconds and 1/2), call startAnimation1(), at 1:20.75 (1 minute, 20 seconds and 3/4), call startAnimation2(), and so on?
switch musicPlayingTimeElapsed {
case 0:52.50:
startAnimation1()
case 1:20.75:
startAnimation2()
default:
()
}
Playing music using AVFoundation
import AVFoundation
var myMusic : AVAudioPlayer?
func playMusic() {
if let musicFile = self.setupAudioPlayerWithFile("fileName", type:"mp3") {
self.myMusic = musicFile
}
myMusic?.play()
}
func setupAudioPlayerWithFile(file:NSString, type:NSString) -> AVAudioPlayer? {
let path = NSBundle.mainBundle().pathForResource(file as String, ofType: type as String)
let url = NSURL.fileURLWithPath(path!)
var audioPlayer:AVAudioPlayer?
do {
try audioPlayer = AVAudioPlayer(contentsOfURL: url)
} catch {
print("AVAudioPlayer not available")
}
return audioPlayer
}
If you use AVPlayer instead of AVAudioPlayer, you can use the (TBH slightly awkward) addBoundaryTimeObserverForTimes method:
let times = [
NSValue(CMTime:CMTimeMake(...)),
NSValue(CMTime:CMTimeMake(...)),
NSValue(CMTime:CMTimeMake(...)),
// etc
];
var observer: AnyObject? = nil // instance variable
self.observer = self.player.addBoundaryTimeObserverForTimes(times, queue: nil) {
switch self.player.currentTime() {
case 0:52.50:
startAnimation1()
case 1:20.75:
startAnimation2()
default:
break
}
}
// call this to stop observer
self.player.removeTimeObserver(self.observer)
The way I solve this is to divide the music up into separate segments beforehand. I then use one of two approaches:
I play the segments one at a time, each in its own audio player. The audio player's delegate is notified when a segment finishes, and so starting the next segment — along with accompanying action — is up to me.
Alternatively, I queue up all the segments onto an AVQueuePlayer. I then use KVO on the queue player's currentItem. Thus, I am notified exactly when we move to a new segment.
You might try using Key Value Observing to observe the duration property of your sound as it plays. When the duration reaches your time thresholds you'd trigger each animation. You'd need to make the time thresholds match times >= the trigger time, since you will likely not get a perfect match with your desired time.
I don't know how well that would work however. First, I'm not sure if the sound player's duration is KVO-compliant.
Next, KVO is somewhat resource-intensive, and if your KVO listener gets called thousands of times a second it might bog things down. It would at least be worth a try.

Swift - Have an audio repeat, but it plays in intervals?

In Swift, I play a song, it plays at the viewDidLoad. It is 16 seconds long, and it plays all 16 seconds. I want it to repeat forever. I made an NSTimer where every 16 seconds, it plays the song. But, it plays 16 seconds when the app loads, stops for 16 seconds, plays, etc.
The line println("just doing some dandy debugging here.") does print every 16 seconds.
How is this fixed?
CODE:
//these var's are created on the top of the file.
var soundTimer: NSTimer = NSTimer()
var audioPlayer2 = AVAudioPlayer()
var soundTwo = NSURL(fileURLWithPath: NSBundle.mainBundle().pathForResource("sound", ofType: "wav"))
//this stuff is in the viewDidLoad function.
audioPlayer2 = AVAudioPlayer(contentsOfURL: soundTwo, error: nil)
audioPlayer2.prepareToPlay()
audioPlayer2.play()
soundTimer = NSTimer.scheduledTimerWithTimeInterval(16, target: self, selector: Selector("soundTimerPlayed"), userInfo: nil, repeats: true)
func soundTimerPlayed() {
println("just doing some dandy debugging here.")
audioPlayer2.stop()
audioPlayer2.prepareToPlay()
audioPlayer2.play()
}
Just do it the easy, official way, instead. From the documentation of the numberOfLoops property:
Set any negative integer value to loop the sound indefinitely until you call the stop method.
But your actual problem is almost certainly because you're stopping playback a little early:
The stop method does not reset the value of the currentTime property to 0. In other words, if you call stop during playback and then call play, playback resumes at the point where it left off.
What's happening is that you're stopping the sound playback just a little before the actual end of the sample—your timer is set to just too short a time. The first time the timer is triggered, the "play" is playing the tiny amount of quiet or silent sound that's left at the end, and then stopping. The next time, the currentTime has been reset because the sample has successfully reached the end of playback, so on the next timer interval, after another 16 seconds, playback starts successfully from the beginning. And repeat.
As Jack observes in his comment, this is why there's a delegate callback to give you a kick when the playback has actually finished, but I'd still just set numberOfLoops and not bother with the complicated stuff, if you only need the sound to loop indefinitely.
(If you desperately want to repeat on your exact timer event, then just set currentTime to 0 before you play it again.)

XNA MediaPlayer loading music timing

I load music with the following code in my load content function:
song = Content.Load<Song>("music/game");
MediaPlayer.IsRepeating = false;
MediaPlayer.Play(song);
nothing strange there, but each round in my game is 2 minutes long and should sync up with the music (that is 2 minutes long) but the music ends betweem 2-4s early. This wouldn't be a problem if it was always the same time.
My guess is that it has something to do with load times? any advice?
One thing you could do is move the Content.Load<Song> to Load method and check if it is playing in the update, and if not, play. Eg,
public void LoadContent(ConentManager content)
{
song = content.Load<Song>("music/game");
gameSongStartedPlaying = false; // this variable to hold if you have starting playing this song already
MediaPlayer.IsRepeating = false;
}
public void Update(GameTime gameTime)
{
if(MediaPlayer.State == MediaState.Stopped && !gameSongStartedPlaying)
{
MediaPlayer.Play(song);
gameSongStartedPlaying = true;
}
}
This should start playing the song on the first pass of the Update method rather than in the Loading phase where the song is 'playing' while all resources after Content.Load<Song> are still loading (this would be the reason your song finishes early).

Resources