I have 2 sequencers:
let sequencer1 = AKAppleSequencer(filename: "filename1")
let sequencer2 = AKAppleSequencer(filename: "filename2")
Both have the same bpm value.
When sequencer1 starts playing one midi track (playing it only once) I need that sequencer2 begin playing exactly after first sequencers finished. How can I achieve this ?
Note that sequencer2 looped.
Currently I have this approach but it is not accurate enough:
let callbackInstrument = AKMIDICallbackInstrument(midiInputName: "callbackInstrument", callback: nil)
let callbackTrack = sequencer1.newTrack()!
callbackTrack.setMIDIOutput(callbackInstrument.midiIn)
let beatsCount = sequencer1.length.beats
callbackTrack.add(noteNumber: MIDINoteNumber(beatsCount),
velocity: 1,
position: AKDuration(beats: beatsCount),
duration: AKDuration(beats: 0.1))
callbackInstrument.callback = { status, _, _ in
guard AKMIDIStatusType.from(byte: status) == .noteOn else { return }
DispatchQueue.main.async { self.sequencer2.play() }//not accurate
}
let sampler = AKMIDISampler(midiOutputName: nil)
sequencer1.tracks[0].setMIDIOutput(sampler.midiIn)
Appreciate any thoughts.
Apple's MusicSequence, upon which AKAppleSequencer is built, always flubs the timing for the first 100ms or so after it starts. It is a known issue in closed source code and won't ever be fixed. Here are two possible ways around it.
Use the new AKSequencer. It might be accurate enough to make this work (but no guarantees). Here is an example of using AKSequencer with AKCallbackInstrument: https://stackoverflow.com/a/61545391/2717159
Use a single AKAppleSequencer, but place your 'sequencer2' content after the 'sequencer1' content. You won't be able to loop it automatically, but you can repeatedly re-write it from your callback function (or pre-write it 300 times or something like that). In my experience, there is no problem writing MIDI to AKAppleSequencer while it is playing. The sample app https://github.com/AudioKit/MIDIFileEditAndSync has examples of time shifting MIDI note data, which could be used to accomplish this.
Related
Hello fellow AudioKit users,
I'm trying to setup AudioKit 5 with a playback time indication, and am having trouble.
If I use AudioPlayer's duration property, this is the total time of the audio file, not the current playback time.
ex:
let duration = player.duration
Always gives the file's total time.
Looking at old code from AKAudioPlayer, it seemed to have a "currentTime" property.
The migration guide (https://github.com/AudioKit/AudioKit/blob/v5-main/docs/MigrationGuide.md) mentions some potentially helpful classes from the old version, however the "AKTimelineTap" has no replacement with no comments from the developers... nice.
Also still not sure how to manipulate the current playback time either...
I've also checked out Audio Kit 5's Cookbooks, however this is for adding effects and nodes, not necessarily for playback display, etc..
Thanks for any help with this new version of AudioKit.
You can find playerNode in AudioPlayer, it's AVAudioPlayerNode class.
Use lastRenderTime and playerTime, you can calculate current time.
ex:
// get playerNode in AudioPlayer.
let playerNode = player.playerNode
// get lastRenderTime, and transform to playerTime.
guard let lastRenderTime = playerNode.lastRenderTime else { return }
guard let playerTime = playerNode.playerTime(forNodeTime: lastRenderTime) else { return }
// use sampleRate and sampleTime to calculate current time in seconds.
let sampleRate = playerTime.sampleRate
let sampleTime = playerTime.sampleTime
let currentTime = Double(sampleTime) / sampleRate
i'm currently working a musician app. In my app notes should be played with a specific duration. I don't get into detail when the notes are played. Basically there is a ui view (a vertical line) which is moving and when this hits my other ui views (rectangle) it should be played a note. Important here: the note should be played until the line is not hitting the rectangle anymore.
The note playing is no problem but I don't find any duration. Also it should be possible to play the same note multiple times with a delay.
So I tried to make this work with AudioKit cause it's seems like the best greatest solution for audio. But it has so much stuff. I took a look into their examples and found this:
let bundlePath = Bundle.main.bundlePath
let soundPath = ("\(bundlePath)/sounds")
let akSampler = AKAppleSampler()
let mixer = AKMixer(akSampler)
try! akSampler.loadSoundFont(soundPath, preset: 0, bank: 0)
mixer.start()
AudioKit.output = mixer
do {
_ = try AudioKit.engine.start()
} catch {
print("AudioKit wouldn't start!")
}
do {
try akSampler.play(noteNumber: myNote.rawValue, velocity: 100, channel: 1)
} catch let e{
print(e)
}
Unfortunately I can't pass any duration and when I call akSampler.stop(noteNumber: myNote.rawValue) it also stops the other notes with the same type.
I tried to find a solution with AVFoundation like so:
engine = AVAudioEngine()
sampler = AVAudioUnitSampler()
engine.attach(sampler)
engine.connect(sampler, to: engine.mainMixerNode, format: nil)
guard let bankURL = Bundle.main.url(forResource: "sounds", withExtension: "SF2") else {
print("could not load sound font")
return
}
... init engine
sampler.startNote(60, withVelocity: 64, onChannel: 0)
But same result. Also the same case that I can't pass any duration.
I also digged into MIDISequencer's but it seems that they generating a sequence which I can play but this does not fit on my problem.
Does someone has a solution here?
The laziest solution would be to just schedule a stop with asyncAfter when you trigger the note, e.g.,
func makeNote(note: MIDINoteNumber, dur: Double) {
sampler.play(noteNumber: note, velocity: 100, channel: 0)
DispatchQueue.main.asyncAfter(deadline: .now() + dur) {
self.sampler.stop(noteNumber: note)
}
}
A better solution would probably use either AKSequencer or AKAppleSequencer. Both allow you to create sequences on the fly by adding individual notes with a specified duration (in musical time, i.e., number of beats). AKSequencer is considerably more accurate, but AKAppleSequencer has more readily available code examples on the web. A little confusingly, the current AKAppleSequencer used to also be called AKSequencer, but their interfaces are sufficiently different that a quick look at the docs for the two classes will tell you which you're looking at.
Your question is asking about how to schedule MIDI events which is precisely what these classes are designed to do. You haven't really given a clear reason why generating a sequence doesn't fit your problem.
I am setting up a tts app with AVSpeechSynthesizer. I have to do real-time pitch and rate adjustments. I am using UISLider for adjusting pitch and rate.
Here is my code:-
#IBAction func sl(_ sender: UISlider) {
if synthesizer.isSpeaking {
synthesizer.stopSpeaking(at: .immediate)
self.rate = sender.value
if currentRange.length > 0 {
let valuee = currentRange.length + currentRange.location
let neww = self.tvEditor.text.dropFirst(valuee)
self.tvEditor.text = String(neww)
synthesizer.speak(buildUtterance(for: rate, pitch: pitch, with: String(neww), language: self.preferredVoiceLanguageCode2 ?? "en"))
}
} else {
}
}
I may have understood your problem even if no details are provided: you can't take into account the new values of the rate and pitchMultiplier when the speech is running.
To explain the following details, I read this example that contains code snippets (ObjC, Swift) and illustrations.
Create your AVSpeechUtterance instances with their rate and pitchMultiplier properties.
Add each one of them in an array that will represent the queue to be spoken.
Make a loop inside the previous queue with the synthesizer to read out every elements.
Now, if you want to change the property values in real-time, see the steps hereafter once one of your sliders moves:
Get the current spoken utterance thanks to the AVSpeechSynthesizerDelegate protocol.
Run the stopSpeaking synthesizer method that will remove from the queue the utterances that haven't been spoken yet.
Create the previous removed utterances with the new property values.
Redo steps 2/ and 3/ to resume where you stopped with these updated values.
The synthesizer queues all information to be spoken long before you ask for new values that don't impact the stored utterances: you must remove and recreate the utterances with their new property values to be spoken.
If the code example provided by the link above isn't enough, I suggest to take a look at this WWDC video detailed summary dealing with AVSpeechSynthesizer.
First, I called a AKMIDISampler to play an audio file, and then assigned it to AKSequencer. The 'midi' file I used is just a 2 bars long, C3 note, single track midi file, exactly as long as the audio file I wanted to play. But, in calling AKAudioFile, I wanted to choose mp3 file randomly. I temporarily made 1.mp3, 2.mp3 and 3.mp3 as below.
let track = AKMIDISampler()
let sequencer = AKSequencer(filename: "midi")
try? track.loadAudioFile(AKAudioFile(readFileName: String(arc4random_uniform(3)+1) + ".mp3"))
sequencer.tracks[0].setMIDIOutput(track.midiIn)
// Tempo track I had to made to remove sine wave
sequencer.tracks[1].setMIDIOutput(track.midiIn)
And did some sequencer settings,
sequencer.setTempo(128.0)
sequencer.setLength(AKDuration(beats: 8))
sequencer.setLoopInfo(AKDuration(beats: 8), numberOfLoops: 4)
sequencer.preroll()
and assigned AKMIDISampler to AudioKit.output, then did sequencer.play().
The sequencer playback was successful! It loaded among three mp3 files randomly, and played 8 beats (2 bars), looped for 4 times exactly.
But my goal is to load random MP3 files every time the loop repeats. It seems like the sequencer only plays the first assigned mp3 file when looping. I am struggling finding a solution to this.
Perhaps I could use "AKCallbackInstrument"? Since I play audiofile through a midi note in this case, I might reset "loadAudioFile" whenever the midi note is off? In that way I might loop the sequencer and play random a audio file in every loop. This is just an idea, but for me now it is hard to write it properly. I hope I am on the right track. It would be great if I could get an advice here. <3
You're definitely on the right track - you can easily get random audio files to loop at a fixed interval with AKSequencer + AKCallbackInstrument. But I wouldn't worry about trying to reload on the NoteOff message.
I would first load each mp3 into a separate player (e.g., AKAppleSampler) in an array (e.g.,you could call it players) and create a method that will trigger one of these players at random:
func playRandom() {
let playerIndex = Int(arc4random_uniform(UInt32(players.count)))
try? players[playerIndex].play()
}
When you create your sequencer, add a track and assign it to an AKCallbackInstrument. The callback function for this AKCallbackInstrument will call playRandom when it receives a noteOn message.
seq = AKSequencer()
track = seq.newTrack()!
callbackInst = AKCallbackInstrument()
track.setMIDIOutput(callbackInst.midiIn)
callbackInst.callback = { status, note, vel in
guard status == .noteOn else { return }
self.playRandom()
}
It isn't necessary to load the sequencer with a MIDI file. You could just add the triggering MIDI event directly to the track.
track.add(noteNumber: 48, // i.e., C3
velocity: 127,
position: AKDuration(beats: 0), // noteOn message here
duration: AKDuration(beats: 8), // noteOff 8 beats later
channel: 0)
Your problem with the sine wave is probably being caused by an extra track (probably tempo track) in the MIDI file which you created which hasn't been assigned an output. You can avoid the problem altogether by adding the MIDI events directly.
In principle, you could use the callback to check for noteOff events and trigger code from the noteOff, but I wouldn't recommend it in your case. There is no good reason to re-use a single player for multiple audiofiles. Loading the file is where you are most likely to create an error. What happens if your file hasn't finished playing and you try to load another one? The resources needed to keep multiple players in memory is pretty trivial - if you're going to play the same file more than once, it is cleaner and safer to load it once and keep the player in memory.
It was very helpful, c_booth! Thanks to you, I made a huge progress today. Here's what I've written based on your advise. First, I made an array of AKPlayers include 6 mp3 files. They're assigned to AKMixer, and then I called sequencer and callback instrument. I made a track and a note on the sequencer, which calls 'playRandom' function on every noteOn :
let players: [AKPlayer] = {
do {
let filenames = ["a1.mp3", "a2.mp3", "a3.mp3", "b1.mp3", "b2.mp3", "b3.mp3"]
return try filenames.map { AKPlayer(audioFile: try AKAudioFile(readFileName: $0)) }
} catch {
fatalError()
}
}()
func playRandom() {
let playerIndex = Int(arc4random_uniform(UInt32(players.count)))
players[playerIndex].play()
}
func addTracks() {
let track = sequencer.newTrack()!
track.add(noteNumber: 48, velocity: 127, position: AKDuration(beats: 0), duration: AKDuration(beats: 16), channel: 0)
track.setMIDIOutput(callbackInst.midiIn)
callbackInst.callback = { status, note, vel in
guard status == .noteOn else { return }
self.playRandom()
}
}
func sequencerSettings() {
sequencer.setTempo(128.0)
sequencer.setLength(AKDuration(beats: 16))
sequencer.setLoopInfo(AKDuration(beats: 16), numberOfLoops: 4)
sequencer.preroll()
}
func makeConnections() {
players.forEach { $0 >>> mixer }
}
func startAudioEngine() {
AudioKit.output = mixer
do {
try AudioKit.start()
} catch {
print(error)
fatalError()
}
}
func startSequencer() {
sequencer.play()
}
This worked great. It randomly selects one from 6 mp3 files (they are all the same length, 128bpm and 16 beats). What I found strange here is, though, the first playback plays two audio files at once. It works fine after the second loop. I changed the numberOfLoop setting, enableLooping(), etc but still the same - plays two files on the first playback. The trackcount is still 1, and I only called one AKPlayer as you could see. Is there anything I can do about this?
Also, ultimately, I'd like to call hundreds of mp3 files on the array, as what I'm trying to make is a sort of DJing app (something like Ableton Live preset). Do you think it's a good idea to use AKPlayer, assuming this code will load mp3 files from the cloud and stream it to the user? Much appreciated. <3
I am having a really difficult time with playing audio in the background of my app. The app is a timer that is counting down and plays bells, and everything worked using the timer originally. Since you cannot run a timer over 3 minutes in the background, I need to play the bells another way.
The user has the ability to choose bells and set the time for these bells to play (e.g. play bell immediately, after 5 minutes, repeat another bell every 10 minutes, etc).
So far I have tried using notifications using DispatchQueue.main and this will work fine if the user does not pause the timer. If they re-enter the app though and pause, I cannot seem to cancel this queue or pause it in anyway.
Next I tried using AVAudioEngine, and created a set of nodes. These will play while the app is in the foreground but seem to stop upon backgrounding. Additionally when I pause the engine and resume later, it won't pause the sequence properly. It will squish the bells into playing one after the other or not at all.
If anyone has any ideas of how to solve my issue that would be great. Technically I could try remove everything from the engine and recreate it from the paused time when the user pauses/resumes, but this seems quite costly. It also doesn't solve the problem of the audio stopping in the background. I have the required background mode 'App plays audio or streams audio/video using Airplay', and it is also checked under the background modes in capabilities.
Below is a sample of how I tried to set up the audio engine. The registerAndPlaySound method is called several other times to create the chain of nodes (or is this done incorrectly?). The code is kinda messy at the moment because I have been trying many ways trying to get this to work.
func setupSounds{
if (attached){
engine.detach(player)
}
engine.attach(player)
attached = true
let mixer = engine.mainMixerNode
engine.connect(player, to: mixer, format: mixer.outputFormat(forBus: 0))
var bell = ""
do {
try engine.start()
} catch {
return
}
if (currentSession.bellObject?.startBell != nil){
bell = (currentSession.bellObject?.startBell)!
guard let url = Bundle.main.url(forResource: bell, withExtension: "mp3") else {
return
}
registerAndPlaySound(url: url, delay: warmUpTime)
}
}
func registerAndPlaySound(url: URL, delay: Double) {
do {
let file = try AVAudioFile(forReading: url)
let format = file.processingFormat
let capacity = file.length
let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(capacity))
do {
try file.read(into: buffer)
}catch {
return
}
let sampleRate = buffer.format.sampleRate
let sampleTime = sampleRate*delay
let futureTime = AVAudioTime(sampleTime: AVAudioFramePosition(sampleTime), atRate: sampleRate)
player.scheduleBuffer(buffer, at: futureTime, options: AVAudioPlayerNodeBufferOptions(rawValue: 0), completionHandler: nil)
player.play()
} catch {
return
}
}