It seems to me that CoreAudio adds sound waves together when mixing into a single channel. My program will make synthesised sounds. I know the amplitudes of each of the sounds. When I play them together should I add them together and multiply the resultant wave to keep within the range? I can do it like this:
MaxAmplitude = max1 + max2 + max3 //Maximum amplitude of each sound wave
if MaxAmplitude > 1 then //Over range
Output = (wave1 + wave2 + wave3)/MaxAmplitude //Meet range
else
Output = (wave1 + wave2 + wave3) //Normal addition
end if
Can I do it this way? Should I pre-analyse the sound waves to find the actual maximum amplitude (Because the maximum points may not match on the timeline) and use that?
What I want is a method to play several synthesised sounds together without reducing the volume throughout extremely and sounding seamless. If I play a chord with several synthesised instruments, I don't want to require single notes to be practically silent.
Thank you.
Changing the scale suddenly on a single sample basis, which is what your "if" statement does, can sound very bad, similar to clipping.
You can look into adaptive AGC (automatic gain control) which will change the scale factor more slowly, but could still clip or get sudden volume changes during fast transients.
If you use lookahead with the AGC algorithm to prevent sudden transients from clipping, then your latency will get worse.
If you do use AGC, then isolated notes may sound like they were played much more loudly than when played in a chord, which may not correctly represent a musical composition's intent (although this type of compression is common in annoying TV and radio commercials).
Scaling down the mixer output volume so that the notes will never clip or have their volume reduced other than when the composition indicates will result in a mix with greatly reduced volume for a large number of channels (which is why properly reproduced classical music on the radio is often too quiet to draw enough viewers to make enough money).
It's all a trade-off.
I don't see this is a problem. If you know the max amplitude of all your waves (for all time) it should work. Be sure not to change the amplitude on per sample basis but decide for every "note-on". It is a very simple algorithm but could suit your needs.
Related
I want to play very basic sounds using the AudioKit oscillator at a given frequency.
For example: play a simple sine wave at 400Hz for 50 miliseconds, then 100 ms of silence, then play 600Hz for 50ms …
I have a metal view where I render some visual stimuli. My intention was to use the basic AKOscillator and the render function of the CADisplayLink to play/stop the sound at some frames.
I tried using oscillator.play() and oscillator.stop() or changing the amplitude with oscillator.amplitude = 0 and oscillator.amplitude = 1 but the result in both cases is a jittering of about 10 ms.
If I first create .wav files and then played them with AKPlayer.play() the timing is correct.
I want the flexibility to use any frequency at any time. How can I do something similar to the first approach? Wrapping the oscillator in an midi instrument is the way to go?
CADisplayLink runs in the UI thread, so will have some sub-frame-time jitter, as the UI thread has lower priority than audio (and perhaps other OS) threads.
The only way I've succeeded in reliable sub-millisecond accurate real-time synthesis of arbitrary frequency audio is to put a sinewave oscillator (or waveform generator or table look-up) inside an Audio Unit callback (RemoteIO for iOS), and count samples, before dynamically inserting the stop/start of the desired sinewave (or other waveform) into the callback buffer(s), starting at the correct sample buffer offset index.
At first, I pre-generated sinewave look-up tables; but later did some profiling; and found that just calling the sin() function with a rotating phase, at audio sample rates, took a barely measurable fraction of a percent of CPU time on any contemporary iOS device. If you do use an incrementing phase, make sure to keep the phase in a reasonable numeric range by occasionally adding/subtracting multiples of 2*pi.
This SO answer addresses how to do a screen capture of a UIView. We need something similar, but instead of a single image, the goal is to produce a video of everything appearing within a UIView over 60 seconds -- conceptually like recording only the layers of that UIView, ignoring other layers.
Our video app superimposes layers on whatever the user is recording, and the ultimate goal is to produce a master video merging those layers with the original video. However, using AVVideoCompositionCoreAnimationTool to merge layers with the original video is very, very, very slow: exporting a 60-second video takes 10-20 seconds.
What we found is combining two videos (i.e., only using AVMutableComposition without AVVideoCompositionCoreAnimationTool) is very fast: ~ 1 second. The hope is to create an independent video of the layers and then combine that with the original video only using AVMutableComposition.
An answer in Swift is ideal but not required.
It sounds like your "fast" merge doesn't involve (re)-encoding frames, i.e. it's trivial and basically a glorified file concatenation, which is why it's getting 60x realtime. I asked about that because your "very slow" export is from 3-6 times realtime, which actually isn't that terrible (at least it wasn't on older hardware).
Encoding frames with an AVAssetWriter should give you an idea of the fastest possible non-trivial export and this may reveal that on modern hardware you could halve or quarter your export times.
This is a long way of saying that there might not be that much more performance to be had. If you think about the typical iOS video encoding use case, which would probably be recording 1920p # 120 fps or 240 fps, your encoding at ~6x realtime # 30fps is in the ballpark of what your typical iOS device "needs" to be able to do.
There are optimisations available to you (like lower/variable framerates), but these may lose you the convenience of being able to capture CALayers.
been programming a waveform generator on my iOS device. I have a second order low pass filter with a very high Q going through it. When I bypass the filter, the sound is clean and doesn't pop. However, when the filter is on, when I release the sound there is a pop. Should I be putting an envelope on the filter or should I be doing something else? It's really annoying and I definitely want the pop to be gone. Looking for any advice! Thanks!
A high-Q low-pass filter will by definition have a large time-constant (remember it's an Infinite Impulse response filter).
A IIR filter is a weighted sum of output samples - the equation below is a simple, generalised example:
y[n] = x[n]*K0 + y[n-1]*K1 + y[n-2]*K2 + y[n-3]*K3....y[n-z]*Kz
When you bypass filter (presumably by routing the signal around it), you will suddenly remove any effect the feedback loop has on the output, which may result in a significant discontinuity in the output signal.
Instead, try inserting zeros into the delay line - this will gradually diminish their contribution to the filter equation over subsequent samples.
Alternatively, you could insert a mixer, that mixes 'dry' and 'wet' signal after the filter and change the proportion mixed together with an envelope.
I have some pure tones (sine waves) that I need to fade in and out and also adjust their volume arbitrarily through a thumbwheel. I generate the PCM data along with a WAV header and then encapsulate it in an AVAudioPlayer which plays back fine at constant volume.
Now, as trivial as this sounds, I'm finding that changing volume rapidly on iOS 7 causes some pretty nasty artefacts. Imagine a tone playing with a slider controlling it's volume. Moving the slider around rapidly will cause the noise I'm describing. It's particularly obvious because the source signal is just a tone. If it were music, things would likely just get lost in the relative noise. Oddly, if I directly instrument the device volume via MPMusicPlayerController instead of trying to control things either through AVAudioPlayer's volume or even at the byte level, I get much smoother results and no artefacts. I suspect the device is doing something when it's volume is adjusted that I am not.I know this kind of quantization noise is an issue in audio processing and I'm wondering if anyone may have some advice.
I've also reproduced the issue using sample level playback via Novocaine. No matter what I do, I can't seem to get smooth, noise-free fade in/out characteristics. Any help would be much appreciated.
Have you checked out AVAudioMix? I've used this for ramping volume with an AVPlayer, and it works incredibly well. The function to ramp volume over a specific time has been particularly helpful for fade in/out effects.
If modifying audio volume at the sample level, make sure to smoothly change the volume, never suddenly from one sample to the next, otherwise the result will have discontinuities, which can be very noisy. e.g. gradually fade in any volume changes over many (perhaps a few dozen) milliseconds worth of samples, using either a linear ramp, or an ease-in-ease-out half-cosine curve, per-sample between level settings. Perhaps one controller is doing this automatically, but the other is not.
I was just reading through the DirectX documentation and encountered something interesting in the page for IDirect3DDevice9::BeginScene :
To enable maximal parallelism between
the CPU and the graphics accelerator,
it is advantageous to call
IDirect3DDevice9::EndScene as far
ahead of calling present as possible.
I've been accustomed to writing my game loop to handle input and such, then draw. Do I have it backwards? Maybe the game loop should be more like this: (semi-pseudocode, obviously)
while(running) {
d3ddev->Clear(...);
d3ddev->BeginScene();
// draw things
d3ddev->EndScene();
// handle input
// do any other processing
// play sounds, etc.
d3ddev->Present(NULL, NULL, NULL, NULL);
}
According to that sentence of the documentation, this loop would "enable maximal parallelism".
Is this commonly done? Are there any downsides to ordering the game loop like this? I see no real problem with it after the first iteration... And I know the best way to know the actual speed increase of something like this is to actually benchmark it, but has anyone else already tried this and can you attest to any actual speed increase?
Since I always felt that it was "awkward" to draw-before-sim, I tended to push the draws until after the update but also after the "present" call. E.g.
while True:
Simulate()
FlipBuffers()
Render()
While on the first frame you're flipping nothing (and you need to set up things so that the first flip does indeed flip to a known state), this always struck me as a bit nicer than putting the Render() first, even though the order of operations are the same once you're under way.
The short answer is yes, this is how it's commonly done. Take a look at the following presentation on the game loop in God of War III on the PS3:
http://www.tilander.org/aurora/comp/gdc2009_Tilander_Filippov_SPU.pdf
If you're running a double buffered game at 30 fps, the input lag will be 1 / 30 ~= 0.033 seconds which is way to small to be detected by a human (for comparison, any reaction time under 0.1 seconds on 100 metres is considered to be a false start).
Its worth noting that on nearly all PC hardware BeginScene and EndScene do nothing. In fact the driver buffers up all the draw commands and then when you call present it may not even begin drawing. They commonly buffer up several frames of draw commands to smooth out frame rate. Usually the driver does things based around the present call.
This can cause input lag when frame rate isn't particularly high.
I'd wager if you did your rendering immediately before the present you'd notice no difference to the loop you give above. Of course on some odd bits of hardware this may then cause issues so, in general, you are best off looping as you suggest above.