I’m trying to get a callback at a given point in an AKPlayer’s file playback (currently, just before the end). I see the Apple docs on addBoundaryTimeObserver(), which would work, but it doesn’t seem to be accessible from AKPlayer (I guess an AVAudioPlayerNode vs AVPlayer thing). Any suggestions? I see a few callbacks in AVAudioPlayerNode… maybe I could determine the buffer based on the desired time and use dataConsumed?
The goal is to trigger another event just before the file finishes playing (there is a callback on completion, but obviously that's too late).
If anybody has done something similar, or knows of something similar (a gist, etc), that would be great.
There's an AudioKit playground called AKPlaygroundLoop that shows you how to call an arbitrary handler periodically, based on CADisplayLink. In your handler you could check the AKPlayer's currentTime and if it's close to the end (say 1 second before) you could trigger whatever event you want.
This is a rough outline:
var player: AKPlayer!
var loop: AKPlaygroundLoop!
func play() {
// ...
playgroundLoop = AKPlaygroundLoop(frequency: 10.0, handler: myHandler)
}
func myHandler() {
if player.currentTime >= player.duration - 1.0 {
// trigger some event
}
}
See also this answer for advice on how to synchronize events with AudioKit.
Related
In AudioKit there is this method for AKPlayer:
#objc dynamic public func play(at audioTime: AVAudioTime?)
I want the same for stop method because I want to be able to stop the player at any time when the user hits the stop button. I am making a music app and I need to stop the sound in X time which is calculated based on BPM and etc.
Here is how I start my AKPlayer:
drums.play(at: AVAudioTime.now() + timeToClosestBeatGrid)
I want the same API with stop:
drums.stop(at: AVAudioTime.now() + timeToClosestBeatGrid) // this api doesnt exist :(((
I tried using endTime property by setting it but it does not seem to do anything...
How may I accomplish this?
PS: I am not looking for a Timer solution this is because a timer is not 100% accurate. I want my stop method to be 100% accurate just like play method
The most accurate way to schedule events in AudioKit is by using AKSequencer. The sequencer can be connected to a callback instrument, which is a node that passes the events to an user-defined function.
In your case, you would add an event at the time where you want the player to stop. In your callback function, you would stop the player as a response to that event.
This is an outline of what should be done:
Create a track to contain the stop event, using AKSequencer's addTrack method. Connect this track to an AKCallbackInstrument. Please see this answer on how to connect an AKCallbackInstrument to an AKSequencer track.
Add the stop event to the track, at the time position where you want the music to stop. As you will be interpreting the event yourself with a callback function, it doesn't really matter what type of event you use. You could simply use a Note On.
In the callback function, stop the player when that event is received.
This is what your callback function would look like:
func stopCallback(status:UInt8, note:MIDINoteNumber, vel:MIDIVelocity) -> () {
guard let status = AKMIDIStatus(byte: status),
let type = status.type,
type == .noteOn else { return }
drums.stop()
}
According to AudioKit documentation, you can try using the schedule(at:) method:
You can call this to schedule playback in the future or the player will call it when play() is called to load the audio data
After the play() method you should declare this schedule(at:) with an offset AVAudioTime.now() + timeToClosestBeatGrid and specify .dataPlayedBack as completion callback type, because this completion is called when (from docs)...
The buffer or file has finished playing
and now (in completion block) you can call drums.stop()
But... If the .stop() method should be called whenever the button is pressed, why not use some form of delay (Timer or DispatchQueue) with the value timeToClosestBeatGrid as the offset?
I’m manually writing a video recorder. Unfortunately it’s necessary if you want to record video and use ARKit at the same time. I’ve got most of it figured out, but now I need to optimize it a bit because my phone gets pretty hot running ARKit, Vision and this recorder all at once.
To make the recorder, you need to use an AVAssetWriter with an AVAssetWriterInput (and AVAssetWriterInputPixelBufferAdaptor). The input has a isReadyForMoreMediaData property you need to check before you can write another frame. I’m recording in real-time (or as close to as possible).
Right now, when ARKit.ARSession gives me a new session I immediately pass it to the AVAssetWriterInput. What I want to do is add it to a queue, and have loop check to see if there’s samples available to write. For the life of me I can’t figure out how to do that efficiently.
I want to just run a while loop like this, but it seems like it would be a bad idea:
func startSession() {
// …
while isRunning {
guard !pixelBuffers.isEmpty && writerInput.isReadyForMoreMediaData else {
continue
}
// process sample
}
}
Can I run this a separate thread from the ARSession.delegateQueue? I don't want to run into issues with CVPixelBuffers from the camera being retained for too long.
I am working on a sort of multiple audio playback project. First, I have 10 mp3 files in a folder. I wanted AKPlayer to play one of these audio files randomly, but in sequence - one after the other. But playing a random file after another random file seems to be tricky. Here's what I've written:
let file = try? AKAudioFile(readFileName: String(arc4random_uniform(9)+1) + ".mp3")
let player = AKPlayer(audioFile: file!)
player1.isLoopiong = true
player.buffering = .always
AudioKit.output = AKPlayer
try? AudioKit.start()
player.start(at: startTime)
This code loops the first chosen random file forever - but I simply wanted to play each random files once. Is there any way I can reload the 'file' so the player starts again when it's done playing? I've tried calling multiple AKPlayers (but calling 10 players must be wrong), if player.isPlaying = false, sequencer, etc, but couldn't exactly figure out how. Apologize for such a newbie question. Thank you so much.
AKPlayer has a completion handler
to be called when Audio is done playing. The handler won’t be called
if stop() is called while playing or when looping from a buffer.
The completion handler type is AKCallback, which is a typealias for () -> Void. If you have some good reason not to use 10 AKPlayers, you could probably use the completion handler to change the file and restart the player. But you could also create an array with 10 AKPlayers, each loaded with a different file, and have a function that selects a player at random for playback (or that cycles through a a pre-shuffled array). The completion handler for each player in the array could call this function, when appropriate. As per the doc quoted above, make sure that the AKPlayer is not looping or else the completion handler won't be called.
yes, you can use the completionHandler of the player to load a new file into the same player when playback finishes. In your completion block:
player.load(url: nextFile)
player.play()
Another approach is to use the AKClipPlayer with 10 clips of a predetermined random order and schedule them in sequence. This method will be the most seamless (if that matters).
I'm trying to create a metronome for iOS in Swift. I'm using a GCD dispatch queue to time an AVAudioPlayer. The variable machineDelay is being used to time the player, but its running slower than the time I'm asking of it.
For example, if I ask for a delay of 1sec, it plays at 1.2sec. 0.749sec plays at about 0.92sec, and 0.5sec plays at about 0.652sec. I could try to compensate by adjusting for this discrepancy but I feel like there's something I'm missing here.
If there's a better way to do this altogether, please give suggestions. This is my first personal project so I welcome ideas.
Here are the various functions that should apply to this question:
func milliseconds(beats: Int) -> Double {
let ms = (60 / Double(beats))
return ms
}
func audioPlayerDidFinishPlaying(player: AVAudioPlayer, successfully flag: Bool) {
if self.playState == false {
return
}
playerPlay(playerTick, delay: NSTimeInterval(milliseconds(bpm)))
}
func playerPlay(player: AVAudioPlayer, delay: NSTimeInterval) {
let machineDelay: Int64 = Int64((delay - player.duration) * Double(NSEC_PER_SEC))
dispatch_after(dispatch_time(DISPATCH_TIME_NOW, machineDelay),dispatch_get_main_queue(), { () -> Void in
player.play()
})
}
I have never really done anything with sound on iOS but I can tell you why you are getting those inconsistent timings.
What happens when you use dispatch_after() is that some timer is set somewhere in the OS and at some point soon after it expires, it puts your block on the queue. "at some point after" is going to be short, but depending on what the OS is doing, it will almost certainly not be close to zero.
The main queue is serviced by the main thread using the run loop. This means your task to play the sound is competing for use of the CPU with all the UI functionality. This means that the chance of it playing the sound immediately is quite low.
Finally, the completion handler will fire at some short time after the sound finishes playing but not necessarily straight away.
All of these little delays add up to the latency you are seeing. Unfortunately, depending on what the device is doing, that latency can vary. This is never going to work for something that needs precise timings.
There are, I think, a couple of ways to achieve what you want. However, audio programming is beyond my area of expertise. You probably want to start by looking at Core Audio. My five minutes of research suggests either Audio Queue Services or OpenAL, but those five minutes are literally everything I know about sound on iOS.
dispatch_after is not intended for sample accurate callbacks.
If you are writing audio applications there is no way to escape, you need to implement some CoreAudio code in one way or another.
It will "pull" specific counts of samples. Do the math (figuratively ;)
I know that there is a limitation in iOS Safari where the audio is not playing until user triggers an interaction. So I have placed the code inside a touchstart event. But unfortunately, I have tried almost every combination, and I couldn't get it to play on iOS Safari.
Here are the things I have tried:
putting the audio load outside the touchstart callback
try adding a gain node
use 0.01 as the start time
and none of the above works in iOS Safari, but they can all play in desktop Chrome and Safari. Here is the link to the gist, you can see the versions where I made the changes (P.S. the click event is used for testing on desktop)
https://gist.github.com/angelathewebdev/32e0fbd817410db5dea1
Sounds play only when currentTime starts to run, but scheduling sounds exactly at currentTime doesn't seem to work. They need to be a little bit into the future (ex: 10ms). You can use the following createAudioContext function to wait until the context is ready to make noise. User action doesn't seem to be required on iPhone, but no such success on iPad just yet.
function createAudioContext(callback, errback) {
var ac = new webkitAudioContext();
ac.createGainNode(); // .. and discard it. This gets
// the clock running at some point.
var count = 0;
function wait() {
if (ac.currentTime === 0) {
// Not ready yet.
++count;
if (count > 600) {
errback('timeout');
} else {
setTimeout(wait, 100);
}
} else {
// Ready. Pass on the valid audio context.
callback(ac);
}
}
wait();
}
Subsequently, when playing a note, don't call .noteOn(ac.currentTime), but do .noteOn(ac.currentTime + 0.01) instead.