Why is my multi-channel mixer no longer playing in iOS 8? - ios

I've written some code to play multi-instrument general MIDI files on iOS. It works fine in iOS 7, but stopped working on iOS 8.
I've stripped it down to its essence here. Instead of creating 16 channels for my multi-channel mixer, I just create one sampler node, and map all the tracks to that channel. It still exhibits the same problem as the multi-sampler version. None of the Audio Toolbox calls return an error code (they all return 0) in iOS 7 or iOS 8. The sequence plays through the speakers in iOS 7, on both the simulator and on iPhone/iPad devices. Run the exact same code on the iOS 8 simulator, or an iPhone/iPad device, and no sound is produced.
If you comment out the call to [self initGraphFromMIDISequence], it plays on iOS 8 with the default sine-wave sound.
#implementation MyMusicPlayer {
MusicPlayer _musicPlayer;
MusicSequence _musicSequence;
AUGraph _processingGraph;
}
- (void)playMidi:(NSURL*)midiFileURL {
NewMusicSequence(&_musicSequence);
MusicSequenceFileLoad(_musicSequence, CFBridgingRetain(midiFileURL), 0, 0);
NewMusicPlayer(&_musicPlayer);
MusicPlayerSetSequence(_musicPlayer, _musicSequence);
[self initGraphFromMIDISequence];
MusicPlayerPreroll(_musicPlayer);
MusicPlayerStart(_musicPlayer);
}
// Sets up an AUGraph with one channel whose instrument is loaded from a sound bank.
// Maps all the tracks of the MIDI sequence onto that channel. Basically this is a
// way to replace the default sine-wave sound with another (single) instrument.
- (void)initGraphFromMIDISequence {
NewAUGraph(&_processingGraph);
// Add one sampler unit to the graph.
AUNode samplerNode;
AudioComponentDescription cd = {};
cd.componentManufacturer = kAudioUnitManufacturer_Apple;
cd.componentType = kAudioUnitType_MusicDevice;
cd.componentSubType = kAudioUnitSubType_Sampler;
AUGraphAddNode(_processingGraph, &cd, &samplerNode);
// Add a Mixer unit node to the graph
cd.componentType = kAudioUnitType_Mixer;
cd.componentSubType = kAudioUnitSubType_MultiChannelMixer;
AUNode mixerNode;
AUGraphAddNode(_processingGraph, &cd, &mixerNode);
// Add the Output unit node to the graph
cd.componentType = kAudioUnitType_Output;
cd.componentSubType = kAudioUnitSubType_RemoteIO; // Output to speakers.
AUNode ioNode;
AUGraphAddNode(_processingGraph, &cd, &ioNode);
AUGraphOpen(_processingGraph);
// Obtain the mixer unit instance from its corresponding node, and set the bus count to 1.
AudioUnit mixerUnit;
AUGraphNodeInfo(_processingGraph, mixerNode, NULL, &mixerUnit);
UInt32 const numChannels = 1;
AudioUnitSetProperty(mixerUnit,
kAudioUnitProperty_ElementCount,
kAudioUnitScope_Input,
0,
&numChannels,
sizeof(numChannels));
// Connect the sampler node's output 0 to mixer node output 0.
AUGraphConnectNodeInput(_processingGraph, samplerNode, 0, mixerNode, 0);
// Connect the mixer unit to the output unit.
AUGraphConnectNodeInput(_processingGraph, mixerNode, 0, ioNode, 0);
// Obtain reference to the audio unit from its node.
AudioUnit samplerUnit;
AUGraphNodeInfo(_processingGraph, samplerNode, 0, &samplerUnit);
MusicSequenceSetAUGraph(_musicSequence, _processingGraph);
// Set the destination for each track to our single sampler node.
UInt32 trackCount;
MusicSequenceGetTrackCount(_musicSequence, &trackCount);
MusicTrack track;
for (int i = 0; i < trackCount; i++) {
MusicSequenceGetIndTrack(_musicSequence, i, &track);
MusicTrackSetDestNode(track, samplerNode);
}
// You can use either a DLS or an SF2 file bundled with your app; both work in iOS 7.
//NSString *soundBankPath = [[NSBundle mainBundle] pathForResource:#"GeneralUserv1.44" ofType:#"sf2"];
NSString *soundBankPath = [[NSBundle mainBundle] pathForResource:#"gs_instruments" ofType:#"dls"];
NSURL *bankURL = [NSURL fileURLWithPath:soundBankPath];
AUSamplerBankPresetData bpdata;
bpdata.bankURL = (__bridge CFURLRef) bankURL;
bpdata.bankMSB = kAUSampler_DefaultMelodicBankMSB;
bpdata.bankLSB = kAUSampler_DefaultBankLSB;
bpdata.presetID = 0;
UInt8 instrumentNumber = 46; // pick any GM instrument 0-127
bpdata.presetID = instrumentNumber;
AudioUnitSetProperty(samplerUnit,
kAUSamplerProperty_LoadPresetFromBank,
kAudioUnitScope_Global,
0,
&bpdata,
sizeof(bpdata));
}
I have some code, not included here, which polls to see if the sequence is still playing, by calling MusicPlayerGetTime on the MusicPlayer instance. In iOS 7, the result of that call each time is the number of seconds that have elapsed since it started playing. In iOS 8, the call always returns 0, which presumably means the MusicPlayer does not start playing the sequence on the call to MusicPlayerStart.
The code above is highly order-dependent -- you have to make certain calls before others; e.g., opening the graph before calling getInfo on a node, and not loading instruments until you've assigned the tracks to channels. I've followed all the advice in other StackOverflow threads, and have verified that getting the order correct makes error codes disappear.
Any iOS MIDI experts know what might have changed between iOS 7 and iOS 8 to make this code stop working?

In iOS 8 Apple introduced a slick Obj-C abstraction of the core audio API - AVAudioEngine.
You should probably check it out. https://developer.apple.com/videos/wwdc/2014/#502

Related

Inter App Audio technology : make effect node and instrument node independent

I am writing an HOST application that uses Core Audio's new iOS 7 Inter App Audio technology. I have managed to get the instruments apps and effects app with the help of Inter-App Audio Examples .
The issue is that the effect node is dependent upon the instrument node. I want to make effect node and instrument node independent.
Here i my Try.
if (desc.componentType == kAudioUnitType_RemoteEffect) {
// if ([self isRemoteInstrumentConnected]) {
if (!_engineStarted) // Check if session is active
[self checkStartOrStopEngine];
if ([self isGraphStarted]) // Check if graph is running and or is created, if so, stop it
[self checkStartStopGraph];
if ([self checkGraphInitialized ]) // Check if graph has been inititialized if so, uninitialize it.
Check(AUGraphUninitialize(hostGraph));
Check (AUGraphAddNode (hostGraph, &desc, &effectNode)); // Add remote instrument
//Disconnect previous chain
// Check(AUGraphDisconnectNodeInput(hostGraph, mixerNode, remoteBus));
//Connect the effect node to the mixer on the remoteBus
Check(AUGraphConnectNodeInput (hostGraph, effectNode, 0, mixerNode, remoteBus));
//Connect the remote instrument node to the effect node on bus 0
Check(AUGraphConnectNodeInput (hostGraph, instrumentNode, 0, effectNode, 0));
//Grab audio units from the graph
Check(AUGraphNodeInfo(hostGraph, effectNode, 0, &effect));
currentUnit = &effect;
}
if (currentUnit) {
Check (AudioUnitSetProperty (*currentUnit, // Set stereo format
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
playerBus,
&stereoStreamFormat,
sizeof (stereoStreamFormat)));
UInt32 maxFrames = 4096;
Check(AudioUnitSetProperty(*currentUnit,
kAudioUnitProperty_MaximumFramesPerSlice,
kAudioUnitScope_Global, playerBus,
&maxFrames,
sizeof(maxFrames)));
[self addAudioUnitPropertyListeners:*currentUnit]; // Add property listeners to audio unit
Check(AUGraphInitialize (hostGraph)); // Initialize the graph
[self checkStartStopGraph]; //Start the graph
}
[_connectedNodes addObject:rau];
but my Application Crashes on this Line --
Check(AUGraphInitialize (hostGraph));
And the Error i got ,
ConnectAudioUnit failed with error
-10860 Initialize failed with error
-10860 error -10860 from AUGraphInitialize (hostGraph)
Note :- I have also Attached screenshot of code portion for better understand.
Edit 1 :-
- (void)createGraph {
// 1
NewAUGraph(&hostGraph);
// 2
AudioComponentDescription iOUnitDescription;
iOUnitDescription.componentType =
kAudioUnitType_Output;
iOUnitDescription.componentSubType =
kAudioUnitSubType_RemoteIO;
iOUnitDescription.componentManufacturer =
kAudioUnitManufacturer_Apple;
iOUnitDescription.componentFlags = 0;
iOUnitDescription.componentFlagsMask = 0;
AUGraphAddNode(hostGraph, &iOUnitDescription, &outNode);
// 3
AUGraphOpen(hostGraph);
// 4
Check(AUGraphNodeInfo(hostGraph, outNode, 0, &outputUnit));
// 5
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2;
format.mSampleRate =
[[AVAudioSession sharedInstance] sampleRate];
format.mFormatID = kAudioFormatLinearPCM;
format.mFormatFlags =
kAudioFormatFlagsNativeFloatPacked |
kAudioFormatFlagIsNonInterleaved;
format.mBytesPerFrame = sizeof(Float32);
format.mBytesPerPacket = sizeof(Float32);
format.mBitsPerChannel = 32;
format.mFramesPerPacket = 1;
AudioUnitSetProperty(mixerUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
1,
&format,
sizeof(format));
AudioUnitSetProperty(mixerUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
0,
&format,
sizeof(format));
CAShow(hostGraph);
}
So the error you're seeing, as per apple docs, is due to The specified node cannot be found.
It looks like you've taken the Apple example app you linked and just deleted a bit to attempt to remove 1 node, but I don't believe its that simple. The documentation of the example clearly states the two nodes are dependent. Just changing the addition of remotes method is not going to be enough, because the host still is attempting to create both, as shown by the error you're seeing.
From this file in the example project, you are only showing the changes you made to addRemoteAU but you need to be making changes to createGraph as well, since that is where the hostGraph is initialized with its nodes. If you initialize the graph with only 1 node, then in addRemoteAU you should stop seeing an error due to a node not being found, since the graph at that point won't expect two nodes (which it does now from it's creation).

AudioUnit generating noise with 8000 Sample rate. Xamarin.ios (Monotouch)

I am using AudioUnit class for recording and playback. During recording i can listen sound. Problem is that when i use sample rate 44100 then it working fine but if i use sample rate 8000 then it generating noise. After recording with 8000 sample rate when i play then there is no noise, there is actual sound.
Means only the time of recording it generate noise with actual sound.
My AudioStreamBasicDescription setting is-
audioStreamDescription.Format = AudioFormatType.LinearPCM;
audioStreamDescription.FormatFlags = AudioFormatFlags.LinearPCMIsSignedInteger |
AudioFormatFlags.LinearPCMIsPacked;
audioStreamDescription.SampleRate = 8000; // 44100;
audioStreamDescription.BitsPerChannel = 16;
audioStreamDescription.ChannelsPerFrame = 1;
audioStreamDescription.BytesPerFrame = (16 / 8);
audioStreamDescription.FramesPerPacket = 1;
audioStreamDescription.BytesPerPacket = audioStreamDescription.BytesPerFrame * audioStreamDescription.FramesPerPacket;
audioStreamDescription.Reserved = 0;
AudioUnit setting is-
public void prepareAudioUnit()
{
// Getting AudioComponent Remote output
_audioComponent = AudioComponent.FindComponent(AudioTypeOutput.Remote);
// creating an audio unit instance
audioUnit = new AudioUnit.AudioUnit(_audioComponent);
// turning on microphone
audioUnit.SetEnableIO(true, AudioUnitScopeType.Input, 1 );
audioUnit.SetEnableIO(true, AudioUnitScopeType.Output, 0 );
// setting audio format
var austat = audioUnit.SetFormat(audioStreamDescription, AudioUnitScopeType.Output, 1);
var austatInput = audioUnit.SetFormat(audioStreamDescription, AudioUnitScopeType.Input, 0);
//audioUnit.SetSampleRate(8000.0f, AudioUnitScopeType.Output, 0);
//audioUnit.SetSampleRate(8000.0f, AudioUnitScopeType.Input, 1);
// setting callback method
audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Input, 0);
audioUnit.Initialize();
}
Now, my main question is how i can remove that noise which is comming with actual sound?
If i am not able to explain properly then please let me know.

Playing Percussion instruments with midi on iOS

I have an app that handles playing multiple midi instruments. Everything works great except for playing percussion instruments. I understand that in order to play percussion in General MIDI you must send the events to channel 10. I've tried a bunch of different things, and I can't figure out how to get it to work, here's an example of how I'm doing it for melodic instruments vs percussion.
// Melodic instrument
MusicDeviceMIDIEvent(self.samplerUnit, 0x90, (UInt8)pitch, 127, 0);
// Percussion Instruments
MusicDeviceMIDIEvent(self.samplerUnit, 0x99, (UInt8)pitch, 127, 0);
The sampler Unit is an AudioUnit and the pitch is given as an int through my UI.
Thanks in advance!
Assuming you have some sort of a General MIDI sound font or similar loaded, you need to set the correct status byte before sending pitch/velocity information. So in case of a Standard MIDI Drum Kit (channel 9), you'd do something like this in Swift:
var status = OSStatus(noErr)
let drumCommand = UInt32( 0xC9 | 0 )
let noteOnCommand = UInt32(0x90 | channel)
status = MusicDeviceMIDIEvent(self._samplerUnit, drumCommand, 0, 0, 0) // set device
status = MusicDeviceMIDIEvent(self._samplerUnit, noteOnCommand, noteNum, velocity, 0) // sends note ON message
No need to undertake anything special for MIDI note off messages.
Ok, so I got it working. I guess the way I load the sound font makes it so the channel stuff doesn't do anything. Instead I had to set the bankMSB property on AUSamplerBankPresetData to kAUSampler_DefaultPercussionBankMSB instead of kAUSampler_DefaultMelodicBankMSB
I added a different font loading method specifically for percussion:
- (OSStatus) loadPercussionWithSoundFont: (NSURL *)bankURL {
OSStatus result = noErr;
// fill out a bank preset data structure
AUSamplerBankPresetData bpdata;
bpdata.bankURL = (__bridge CFURLRef) bankURL;
bpdata.bankMSB = kAUSampler_DefaultPercussionBankMSB;
bpdata.bankLSB = kAUSampler_DefaultBankLSB;
bpdata.presetID = (UInt8) 32;
// set the kAUSamplerProperty_LoadPresetFromBank property
result = AudioUnitSetProperty(self.samplerUnit,
kAUSamplerProperty_LoadPresetFromBank,
kAudioUnitScope_Global,
0,
&bpdata,
sizeof(bpdata));
// check for errors
NSCAssert (result == noErr,
#"Unable to set the preset property on the Sampler. Error code:%d '%.4s'",
(int) result,
(const char *)&result);
return result;
}

iOS equalizer by kAudioUnitSubType_NBandEQ

I want to implement my equalizer on iOS device, and it seems to be achievable by kAudioUnitSubType_NBandEQ. However, after I set the gain of each band, I heard nothing changed on the sound.
I use the sample code of BandEQDemo(https://github.com/springlo/BandEQDemo)
OSStatus TOAUGraphAddNode(OSType inComponentType, OSType inComponentSubType, AUGraph inGraph, AUNode *outNode)
{
AudioComponentDescription desc;
desc.componentType = inComponentType;
desc.componentSubType = inComponentSubType;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
return AUGraphAddNode(inGraph, &desc, outNode);
}
TOAUGraphAddNode(kAudioUnitType_Effect,
kAudioUnitSubType_NBandEQ,
graph,
&eqNode);
AUGraphNodeInfo(graph,
eqNode,
NULL,
&equalizerUnit);
// #[ #32, #64, #128, #256, #512, #1025, #2048, #4096, #8192, #16384 ]
Then after I set the gain and check again by these two functions:
- (AudioUnitParameterValue)gainForBandAtPosition:(NSUInteger)bandPosition
{
AudioUnitParameterValue gain;
AudioUnitParameterID parameterID = kAUNBandEQParam_Gain + bandPosition;
TOThrowOnError(AudioUnitGetParameter(equalizerUnit,
parameterID,
kAudioUnitScope_Global,
0,
&gain));
return gain;
}
- (void)setGain:(AudioUnitParameterValue)gain forBandAtPosition:(NSUInteger)bandPosition
{
AudioUnitParameterID parameterID = kAUNBandEQParam_Gain + bandPosition;
TOThrowOnError(AudioUnitSetParameter(equalizerUnit,
parameterID,
kAudioUnitScope_Global,
0,
gain,
0));
NSLog(#"After set gain#[%d]->%f", bandPosition, [self gainForBandAtPosition:bandPosition]);
}
When I get the gain value of each band after I set it(-96dB ~ 24dB), it does correspond to the value I set. But I cannot hear any different on the sound.
Hi fellow audio enthusiast!
I am also creating an app with an equaliser feature and have stumbled upon this web page:
http://www.deluge.co/?q=content/coreaudio-iphone-creating-graphic-equalizer
With a little of cross-referencing, I have found out that the code from your Git source lacks some key features. After line 142, additional parameter set up must be made.
For example, setNumBands method is not utilised, neither is setBands. Also, make sure to include the bypass setting loop - it really won't work without it! Most importantly, check what pieces of code is lacking in your implementation, based on the ling above.
I am sorry I can't give you more through explanation; I have started to learn Audio Units a few days ago; but your question (as well as the BandEQ project) and the page I stumbled upon have solved some of my problems, so I am trying to help you as well.
Nevertheless, this is an official guide for Audio Units and do study it, as so will I! Good luck!

Playing multiple files with a single file player audio unit

I'm trying to use a file player audio unit (kAudioUnitSubType_AudioFilePlayer) to play multiple files (not at the same time, of course). That's on iOS.
So I've successfully opened the files and stored their details in an array of AudioFileID's that I set to the audio unit using kAudioUnitProperty_ScheduledFileIDs. Now I would like to define 2 ScheduledAudioFileRegion's, one per file, and used them with the file player...
But I can't seem to find out:
How to set the kAudioUnitProperty_ScheduledFileRegion property to store these 2 regions (actually, how to define the index of each region)?
How to trigger the playback of a specific region.. My guess is that the kAudioTimeStampSampleTimeValid parameter should enable this but how to define which region you want to play?
Maybe I'm just plain wrong about the way I should use this audio unit, but documentation is very difficult to get and I haven't found any example showing the playback of 2 regions on the same player!
Thanks in advance.
You need to schedule region every time you want play file. In ScheduledAudioFileRegion you must set AudioFileID to play. Playback begins when current time in unit (samples) are equal or greater than sample time in scheduled region.
Example:
// get current unit time
AudioTimeStamp timeStamp;
UInt32 propSize = sizeof(AudioTimeStamp);
AudioUnitGetProperty(m_playerUnit, kAudioUnitProperty_CurrentPlayTime, kAudioUnitScope_Global, 0, &timeStamp, &propSize);
// when to start playback
timeStamp.mSampleTime += 100;
// schedule region
ScheduledAudioFileRegion region;
memset(&region, 0, sizeof(ScheduledAudioFileRegion));
region.mAudioFile = ...; // your AudioFileID
region.mFramesToPlay = ...; // count of frames to play
region.mLoopCount = 1;
region.mStartFrame = 0;
region.mTimeStamp = timeStamp;
AudioUnitSetProperty(m_playerUnit, kAudioUnitProperty_ScheduledFileRegion, kAudioUnitScope_Global, 0, &region,sizeof(region));

Resources