AudioKit include a great tool to track signal amplitude: AKAmplitudeTracker
This tracker can be init with a thresholdCallback, I suppose that the callback should trigger when the threshold is reach.
I'm playing with the MicrophoneAnalysis example and I can't find a way to trigger my callback.
Here is my code:
var mic: AKMicrophone!
var trackerAmplitude: AKAmplitudeTracker!
var silence: AKBooster!
AKSettings.audioInputEnabled = true
mic = AKMicrophone()
trackerAmplitude = AKAmplitudeTracker(mic, halfPowerPoint: 10, threshold: 0.01, thresholdCallback: { (success) in
print("thresholdCallback: \(success)")
})
trackerAmplitude.start()
silence = AKBooster(trackerAmplitude, gain: 0)
AudioKit.output = silence
I tried to play with the halfPowerPoint and threshold values, but even with vey low values I cannot find a way to print anything :/
Whereas when I'm printing trackerAmplitude.amplitude, I've got values higher than 0.01
Is there something I'm missing ?
The following code works. Tested with AudioKit 4.9, Xcode 11.2, macOS Playground.
This might be an issue of AudioKit, but threshold must be changed via property to activate tracking, as shown below...
import AudioKitPlaygrounds
import AudioKit
let mic = AKMicrophone()
AKSettings.audioInputEnabled = true
let amplitudeTracker = AKAmplitudeTracker(mic, halfPowerPoint: 10, threshold: 1, thresholdCallback: { (success) in
print("thresholdCallback: \(success)")
})
AudioKit.output = amplitudeTracker
try AudioKit.start()
amplitudeTracker.threshold = 0.01 // !! MUST BE SET VIA PROPERTY
amplitudeTracker.start()
mic?.start()
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
Related
EDIT #2: OK, I missed something big here, but I still have a problem. The reason the sound is soft and I have to amplify it is that it is coming from the earpiece, not the speaker. When I add the option .defaultToSpeaker to the setCategory I get no sound at all.
So, this is the real problem, when I set the category to .playbackAndRecord and the option to .defaultToSpeaker, why do I get no sound at all on a real phone? In addition to no sound, I did not receive input from the mic either. The sound is fine in the simulator.
EDIT #3: I began observing route changes and my code reports the following when the .defaultToSpeaker option is included.
2020-12-26 12:17:56.212366-0700 SST[13807:3950195] Current route:
2020-12-26 12:17:56.213275-0700 SST[13807:3950195] <AVAudioSessionRouteDescription: 0x2816af8e0,
inputs = (
"<AVAudioSessionPortDescription: 0x2816af900, type = MicrophoneBuiltIn; name = iPhone Microphone; UID = Built-In Microphone; selectedDataSource = Bottom>"
);
outputs = (
"<AVAudioSessionPortDescription: 0x2816af990, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>
The output is set to Speaker. Is it significant that the selectedDataSource is (null)? Before the .defaultToSpeaker option was added this reported output set to Receiver, also with selectedDataSource = (null), so I would guess not.
EDIT: I added the code to set the Audio Session category. The new code is shown below. So far it seems to have no effect. If I leave it in or comment it out, I don't see any difference. I also have code (that I deleted here for simplicity) that modifies the microphone pattern. That too had no discernible effect. Perhaps though, that is to be expected?
In addition to the symptoms below, if I use Settings/Bluetooth to select the AirPods, then I got no output from the App, even after I remove the AirPods.
What am I missing here?
/EDIT
After getting this to work well on the simulator, I moved to debugging on my 11 Pro Max. When playing notes on the MandolinString, the sound from the (11 Pro Max or an 8) simulator is loud and clear. On the real phone, the sound is barely audible and from the speaker only. It does not go to the attached audio speaker, be that a HomePod or AirPods. Is this a v5 bug? Do I need to do something with the output?
A second less important issue is that when I instantiate this object the MandolinString triggers without me calling anything. The extra fader and the reset of the gain from 0 to 1 after a delay suppresses this sound.
private let engine = AudioEngine()
private let mic : AudioEngine.InputNode
private let micAmp : Fader
private let mixer1 : Mixer
private let mixer2 : Mixer
private let silence : Fader
private let stringAmp : Fader
private var pitchTap : PitchTap
private var tockAmp : Fader
private var metro = Timer()
private let sampler = MIDISampler(name: "click")
private let startTime = NSDate.timeIntervalSinceReferenceDate
private var ampThreshold: AUValue = 0.12
private var ampJumpSize: AUValue = 0.05
private var samplePause = 0
private var trackingNotStarted = true
private var tracking = false
private var ampPrev: AUValue = 0.0
private var freqArray: [AUValue] = []
init() {
// Set up mic input and pitchtap
mic = engine.input!
micAmp = Fader(mic, gain: 1.0)
mixer1 = Mixer(micAmp)
silence = Fader(mixer1, gain: 0)
mixer2 = Mixer(silence)
pitchTap = PitchTap(mixer1, handler: {_ , _ in })
// All sound is fed into mixer2
// Mic input is faded to zero
// Now add String sound to Mixer2 with a Fader
pluckedString = MandolinString()
stringAmp = Fader(pluckedString, gain: 4.0)
mixer2.addInput(stringAmp)
// Create a sound for the metronome (tock), add as input to mixer2
try! sampler.loadWav("Click")
tockAmp = Fader(sampler, gain: 1.0)
mixer2.addInput(tockAmp)
engine.output = mixer2
self.pitchTap = PitchTap(micAmp,
handler:
{ freq, amp in
if (self.samplePause <= 0 && self.tracking) {
self.samplePause = 0
self.sample(freq: freq[0], amp: amp[0])
}
})
do {
//try audioSession.setCategory(AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.measurement)
try audioSession.setCategory(AVAudioSession.Category.playAndRecord)
//, options: AVAudioSession.CategoryOptions.defaultToSpeaker)
try audioSession.setActive(true)
} catch let error as NSError {
print("Unable to create AudioSession: \(error.localizedDescription)")
}
do {
try engine.start()
akStartSucceeded = true
} catch {
akStartSucceeded = false
}
} // init
XCode 12, iOS 14, SPM. Everything up to date
Most likely this is not an AudioKit issue per se, it has to do with AVAudioSession, you probably need to set it on the device to be DefaultToSpeaker. AudioKit 5 has less automatic session management compared to version 4, opting to make fewer assumptions and let the developer have control.
The answer was indeed to add code for AVAudioSession. However, it did not work where I first put it. It only worked for me when I put it in the App delegate didFInishLauchWithOptions. I found this in the AudioKit Cookbook. This works:
class AppDelegate: UIResponder, UIApplicationDelegate {
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
// Override point for customization after application launch.
#if os(iOS)
self.audioSetup()
#endif
return true
}
#if os(iOS)
func audioSetup() {
let session = AVAudioSession.sharedInstance()
do {
Settings.bufferLength = .short
try session.setPreferredIOBufferDuration(Settings.bufferLength.duration)
try session.setCategory(.playAndRecord,
options: [.defaultToSpeaker, .mixWithOthers])
try session.setActive(true)
} catch let err {
print(err)
}
// Other AudioSession stuff here
do {
try session.setActive(true)
} catch let err {
print(err)
}
}
#endif
}
I'm on AudioKit 4.9.1 and can't manage to play a MIDI file with the new AKSequencer (replacing AKAppleSequencer). No sound playing. Assume that MIDI file AND samples are loaded correctly since they previously worked with AKAppleSequencer. Background audio mode capability is also enabled.
Here's the relevant code: (I've also tried both AKSampler and AKAppleSampler but same result)
class MIDIPlayer {
var sampler: AKSampler
var legacySampler: AKAppleSampler
var sequencer: AKSequencer
init(withSfz sfz: String, orSf2 sf2: String, andMidiFile midiFile: String) {
self.sampler = AKSampler()
self.legacySampler = AKAppleSampler()
try? legacySampler.loadSoundFont(sf2, preset: 0, bank: 0)
sampler.loadSFZ(url: Bundle.main.url(forResource: sfz, withExtension: "sfz")!)
AudioKit.output = sampler
try? AudioKit.start()
sequencer = AKSequencer(targetNode: sampler)
// sequencer = AKSequencer(targetNode: legacySampler)
let midi = AKMIDIFile(url: Bundle.main.url(forResource: midiFile, withExtension: "mid")!)
sequencer.load(midiFile: midi)
}
func play() {
sequencer.playFromStart()
}
Is there some difference in how to set up the signal chain that I'm missing?
With the new sequencer, it has to be part of the signal chain. So, do something like
let mixer = AKMixer
sampler >>> mixer
for track in sequencer.tracks { track >>> mixer }
AudioKit.output = mixer
and it should work. Sorry for the delay in seeing this on Github issues.
I can't seem to get the AudioKit instruments to behave the way I'd like: I want to be able to change the frequency continuously and also have the instruments play for an infinite amount of time, just like the oscillators. However, I can't even get a simple playground like the following to output any sound:
//: ## Flute
//: Physical model of a Flute
import AudioKitPlaygrounds
import AudioKit
let playRate = 2.0
let flute = AKFlute()
let reverb = AKReverb(flute)
var triggered = false
let performance = AKPeriodicFunction(frequency: playRate) {
if !triggered {
flute.frequency = 240.0
flute.amplitude = 0.6
flute.play()
triggered = true
}
}
AudioKit.output = reverb
try AudioKit.start(withPeriodicFunctions: performance)
performance.start()
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
The behavior I want is the ability to set the frequency at any time and have the note ring-out forever. Is this possible?
Change flute.play() to flute.trigger()
I’m having a problem using AKMIDISampler in my project when there’s an initialized AKMicrophone. Along with correctly playing the woodblock sample when “play” is called on the sampler, the first time “play” is called a constant sine wave starts playing - it never stops.
I’ve replicated the problem in the smallest amount of code below. Issue happens when the class is initialized then playTestSample() is called.
Note that if the AKMicrophone related code is all muted the AKMIDISampler plays fine and the sine wave that currently haunts my dreams doesn’t happen.
(I’ve tried switching to use the AKSampler() just to see if the problem would exist there but I haven’t been able to get any sound out of that).
Fyi: I have “App plays audio or streams audio/video using AirPlay” in the “Required background modes” in info.plist - which is know to fix another sine wave issue.
Thank you very much for any assistance.
Btw: AudioKit rocks and has been a massive help on this project! :^)
AK 4.5.4
Xcode 10.1
import Foundation
import AudioKit
class AudioKitTESTManager {
var mixer = AKMixer()
var sampler = AKMIDISampler()
var mic = AKMicrophone()
var micMixer = AKMixer()
var micBooster = AKBooster()
init() {
mixer = AKMixer(sampler, micBooster)
do {
let woodblock = try AKAudioFile(readFileName: RhythmGameConfig.woodblockSoundName)
try sampler.loadAudioFiles([woodblock])
} catch {
print("Error loading audio files into sampler")
}
micMixer = AKMixer(mic)
micBooster = AKBooster(micMixer)
micBooster.gain = 0.0
AudioKit.output = mixer
AKSettings.playbackWhileMuted = true
AKSettings.defaultToSpeaker = true
AKSettings.sampleRate = 44100
do {
print("Attempting to start AudioKit")
try AudioKit.start()
} catch {
AKLog("AudioKit did not start!")
}
}
func playTestSample() {
// You hear the sample and a continuous sine wave starts playing through the samplerMixer
try? sampler.play(noteNumber: 60, velocity: 90, channel: 1)
}
}
Wheeew. I believe I've found a solution. Maybe it will help out someone else?
It seems that loading the files into the sampler AFTER AudioKit.start() fixes the Sine Wave of Terror!
//..
do {
print("Attempting to start AudioKit")
try AudioKit.start()
} catch {
AKLog("AudioKit did not start!")
}
do {
let woodblock = try AKAudioFile(readFileName: RhythmGameConfig.woodblockSoundName)
try sampler.loadAudioFiles([woodblock])
} catch {
print("Error loading audio files into sampler")
}
I am trying to play a short musical note sequence with a default sine wave as sound inside a Swift Playground. At a later point I'd like to replace the sound with a Soundfont but at the moment I'd be happy with just producing some sound.
I want this to be a midi like sequence with direct control over the notes, not something purely audio based. The AudioToolbox seems to provide what I am looking for but I have troubles fully understanding its usage. Here's what I am currently trying
import AudioToolbox
// Creating the sequence
var sequence:MusicSequence = nil
var musicSequence = NewMusicSequence(&sequence)
// Creating a track
var track:MusicTrack = nil
var musicTrack = MusicSequenceNewTrack(sequence, &track)
// Adding notes
var time = MusicTimeStamp(1.0)
for index:UInt8 in 60...72 {
var note = MIDINoteMessage(channel: 0,
note: index,
velocity: 64,
releaseVelocity: 0,
duration: 1.0 )
musicTrack = MusicTrackNewMIDINoteEvent(track, time, ¬e)
time += 1
}
// Creating a player
var musicPlayer:MusicPlayer = nil
var player = NewMusicPlayer(&musicPlayer)
player = MusicPlayerSetSequence(musicPlayer, sequence)
player = MusicPlayerStart(musicPlayer)
As you can imagine, there's no sound playing. I appreciate any ideas on how to have that sound sequence playing aloud.
You have to enable the asynchronous mode for the Playground.
Add this at the top (Xcode 7, Swift 2):
import XCPlayground
XCPlaygroundPage.currentPage.needsIndefiniteExecution = true
and your sequence will play.
The same for Xcode 8 (Swift 3):
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
Working MIDI example in a Swift Playground
import PlaygroundSupport
import AudioToolbox
var sequence : MusicSequence? = nil
var musicSequence = NewMusicSequence(&sequence)
var track : MusicTrack? = nil
var musicTrack = MusicSequenceNewTrack(sequence!, &track)
// Adding notes
var time = MusicTimeStamp(1.0)
for index:UInt8 in 60...72 { // C4 to C5
var note = MIDINoteMessage(channel: 0,
note: index,
velocity: 64,
releaseVelocity: 0,
duration: 1.0 )
musicTrack = MusicTrackNewMIDINoteEvent(track!, time, ¬e)
time += 1
}
// Creating a player
var musicPlayer : MusicPlayer? = nil
var player = NewMusicPlayer(&musicPlayer)
player = MusicPlayerSetSequence(musicPlayer!, sequence)
player = MusicPlayerStart(musicPlayer!)
PlaygroundPage.current.needsIndefiniteExecution = true
Great MIDI reference page with a nice chart