I have a fairly complex app that has been working with the AKAppleSequencer up until now, but due to some strange behavior and bugs that pop up now and then with that sequencer, I've been hoping to move to the newer AKSequencer. Unfortunately, the new sequencer doesn't seem to be represented in the Playgrounds or much documentation, so I have been doing some guesswork. I have everything wired up in a way that seems to make sense (to me) and, as I mentioned, was working fine with AKAppleSequencer, but with AKSequencer it runs but no output is produced.
The structure of my code is broken out into multiple pieces so the node graph gets built up in disparate locations, so I'll have to show it here in chunks, with irrelevant lines deleted.
// This happens during setup
mainMixer = AKMixer()
mainMixer.volume = volume
AudioKit.output = mainMixer
// In later code, the sequencer is constructed
sequencer = AKSequencer()
sequencer!.tempo = tempo
// After the sequencer is created, I create various nodes and tracks, like this
let trackNode = trackDefinition.createNode()
let track = sequencer.addTrack(for: trackNode)
track >>> mainMixer
There's a line up there where I'm calling "createNode()" on a thing called trackDefinition. I don't think the details of that class are relevant here, but here's an example of the body of that method's code. It's pretty straightforward.
func createNode() -> AKNode {
let pad = AKMIDISampler()
do {
try pad.loadSoundFont(partConfiguration.settings["soundFontName"]!,
preset: Int(partConfiguration.settings["preset"]!)!,
bank: Int(partConfiguration.settings["bank"]!)!)
} catch {
print("Error while loading Sound Font in PadTrackDefinition: \(error)")
}
return pad
}
That code seems to be working fine. I just wanted to illustrate that I'm creating an AKMIDISampler node, loading a soundfont, and then using that node to create a track in the AKSequencer. Then I attach the track to the main mixer for output.
I used AudioKit.printConnections() to get some confirmation, and here's what that looks like.
(1]AUMultiChannelMixer <2 ch, 44100 Hz, Float32, non-inter> -> (0]AudioDeviceOutput) bus: 0
(2]Local AKSequencerTrack <2 ch, 44100 Hz, Float32, non-inter> -> (1]AUMultiChannelMixer) bus: 0
Pretty simple... Track >>> Mixer >>> Output
Doesn't make any sound when playing.
I also tried it this way:
(0]AUSampler <2 ch, 44100 Hz, Float32, non-inter> -> (2]AUMultiChannelMixer) bus: 0
(2]AUMultiChannelMixer <2 ch, 44100 Hz, Float32, non-inter> -> (1]AudioDeviceOutput) bus: 0
So that's AKMIDISampler >>> Mixer >>> Output (and the sampler was used to create a track).
That also doesn't make any sound.
I also saw this answer to a similar question on StackOverflow, so I tried that approach. That gave me this connection graph:
(0]AUMultiChannelMixer <2 ch, 44100 Hz, Float32, non-inter> -> (1]AudioDeviceOutput) bus: 0
(2]Local AKSequencerTrack <2 ch, 44100 Hz, Float32, non-inter> -> (0]AUMultiChannelMixer) bus: 0
(3]AUSampler <2 ch, 44100 Hz, Float32, non-inter> -> (0]AUMultiChannelMixer) bus: 1
That would be [AKMIDISampler, Track] >>> Mixer >>> Output.
Still...no sound.
What am I doing wrong here? Is there some more specific way that the new sequencer tracks have to be connected into the signal graph that I'm not understanding?
UPDATE: Weird/fun/interesting addendum, if I add this code immediately after the node construction code, it produces the expected note, so I know that at least the audio engine itself is hooked up:
let midiNode = trackNode as! AKMIDISampler
try! midiNode.play(noteNumber: 60,
velocity: MIDIVelocity(127),
channel: MIDIChannel(8))
I figured this out, and wanted to post the answer here for future developers who may run into confusion around this, and also for the core AudioKit team to see, so they can understand what might not be obvious from the API.
The root of the problem here was that the AKSequencer is not a drop-in replacement for the AKAppleSequencer, even though the APIs for the two are extremely similar.
One thing to point out: I have confirmed that it is in fact necessary to add both the track itself and the track's target node to the signal chain in order to get sound output. So from my examples above, you need this one:
[AKMIDISampler, Track] >>> Mixer >>> Output
This is sort of weird and confusing, because it's not at all obvious where I would be expected to put effects nodes in between those. I haven't played with that yet, but it seems very strange to have these nodes both be siblings in the signal chain. I would think it would look like this:
Track >>> AKMIDISampler >>> Mixer >>> Output
That makes more sense to me. Oh well.
Anyway, I mentioned that there were some other factors that were the root of the problem. The key difference was that with the AKAppleSequencer, the track lengths could start out at 0 and then grow as you added additional notes to them. This is the approach I was using, as I was starting with empty tracks and then populating them procedurally.
With the new AKSequencer, it doesn't appear to work that way. The length starts out as 4.0, not 0, and it does not grow automatically as you add notes to the tracks. I had to manually calculate the length required to fit my notes, and then set that length using track.length = desiredLength. The good news is, the AKSequencer is able to understand to use the length of the track, so you can set it on just the tracks, and not the sequencer itself if you prefer.
Another notable difference is the behavior of stop() on the sequencer. On the AKAppleSequencer, invoking stop() also stops the playback of all the notes. On the new AKSequencer, the same method will leave notes playing. You need to do a loop over the tracks like this:
sequencer.stop()
for track in sequencer.tracks {
track.stopPlayingNotes()
}
I know the AKSequencer is brand new, so some things like this are to be expected. I still have hope that it is going to be better in the long run than the AKAppleSequencer.
I hope this explanation will help out somebody like me who got stuck switching to the new sequencer!
I want to specify a duration for example 4 min when using ffmpeg but i keep getting error:
ffmpeg -i "./test.m3u8" -t 04:00 "output.mp4"
and error i get is this:
Invalid duration specification for t: 04:00
also these warnings in yellow color:
max_analyze_duration 5000000 reached at 5014800
Could not find codec parameters (Video: h264 (
Could not find codec parameters (Audio: aac (
Could not find codec parameters (Video: h264 (
Could not find codec parameters (Audio: aac (
hope you guys help me what i am doing wrong. Thanks in advance.
From the documentation:
duration may be a number in seconds, or in hh:mm:ss[.xxx] form.
Your form is mm:ss, so it's simply not valid. Which is also what the error message says.
Use -t 00:04:00 or -t 240 instead.
I've specified and instantiated two Audio Units: a multichannel mixer unit and a generator of subtype AudioFilePlayer.
I would have thought I needed to set the ASBD of the filePlayer's output to match the ASBD I set for the mixer input. However when I attempt to set the filePlayer's output I get a kAudioUnitErr_FormatNotSupported (-10868) error.
Here's the stream format I set on the mixer input (successfully) and am also trying to set on the filePlayer (it's the monostream format copied from Apple's mixerhost sample project):
Sample Rate: 44100
Format ID: lpcm
Format Flags: C
Bytes per Packet: 2
Frames per Packet: 1
Bytes per Frame: 2
Channels per Frame: 1
Bits per Channel: 16
In the course of troubleshooting this I queried the filePlayer AU for the format it is 'natively' set to. This is what's returned:
Sample Rate: 44100
Format ID: lpcm
Format Flags: 29
Bytes per Packet: 4
Frames per Packet: 1
Bytes per Frame: 4
Channels per Frame: 2
Bits per Channel: 32
All the example code I've found sends the output of the filePlayer unit to an effect unit and set the filePlayer's output to match the ASBD set for the effect unit. Given I have no effect unit it seems like setting the filePlayer's output to the mixer input's ASBD would be the correct - and required - thing to do.
How have you configured the AUGraph? I might need to see some code to help you out.
Setting the output scope of AUMultiChannelMixer ASBD once only (as in MixerHost) works. However if you have any kind of effect at all, you will need to think about where their ASBDs are defined and how you arrange your code so CoreAudio does not jump in and mess with your effects AudioUnits ASBDs. By messing with I mean overriding your ASBD to the default kAudioFormatFlagIsFloat, kAudioFormatFlagIsPacked, 2 channels, non-interleaved. This was a big pain for me at first.
I would set the effects AudioUnits to their default ASBD. Assuming you have connected the AUFilePlayer node, then you can pull it out later in the program like this
result = AUGraphNodeInfo (processingGraph,
filePlayerNode,
NULL,
&filePlayerUnit);
And then proceed to set
AudioUnitSetProperty(filePlayerUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
0,
&monoStreamFormat,
sizeof(monoStreamFormat));
Hopefully this helps.
Basically I didn't bother setting the filePlayer ASBD but rather retrieved the 'native' ASBD it was set to and updated only the sample rate and channel count.
Likewise I didn't set input on the mixer and let the mixer figure it's format out.
I've got an MKV file whose source is interlaced NTSC MPEG-2 video. I converted it to H.264 MKV with HandBrake, but this process did not set the "interlaced" flag in the MKV file. The content is interlaced—and I do want it to stay interlaced because it looks much better playing back as 60 fields-per-second content with on-the-fly deinterlacing than it does as 30 frames-per-second content that's been deinterlaced at encode-time.
I tried this...
mkvpropedit -e track:v1 -a interlaced=1 foo.mkv
which did indeed set the interlaced bit...
|+ Segment tracks
| + A track
| + Video track
| + Pixel width: 704
| + Pixel height: 480
| + Display width: 625
| + Display height: 480
| + Interlaced: 1
But when I play the video with VLC with Deinterlace set to Automatic, it doesn't think the video is interlaced and therefore doesn't do the deinterlacing.
What am I doing wrong?
Software versions:
HandBrake 0.9.5
mkvpropedit v5.0.1
Mac OS X 10.7.3
to make handbrake set the interlaced flag:
-use H.264(x264) Video Codec
-at the bottom of the Advanced Tab add :tff or :bff, ( dependant if source is top field first or bottom field first)
I would recommend trying FFMPEG.
Documentation: http://ffmpeg.org/ffmpeg.html
‘-ilme’
Force interlacing support in encoder (MPEG-2 and MPEG-4 only).
Use this option if your input file is interlaced and you want to keep
the interlaced format for minimum losses. The alternative is to
deinterlace the input stream with ‘-deinterlace’, but deinterlacing
introduces losses.
Since you mentioned you are on OSX 10.7 you can use MacPorts to install all dependencies + ffmpeg for you (once the deps are installed you can also build the latest from git).
http://www.macports.org/
(You must be comfortable with the command line for all these tools.)
My app uses ffmpeg to parse mms streaming to buffers, and uses Audio queue to play the buffered data. Now I can confirm the buffered data is good, but I can not play it. The structure of playing is based on the AudioFileStreamExample demo's client, and it can play the http stream(AudioFileStreamExample demo's server). I notice in the callback function:
void MyPacketsProc(void *inClientData, UInt32 inNumberBytes,
UInt32 inNumberPackets, const void *inInputData,
AudioStreamPacketDescription *inPacketDescriptions),
When I directly play the AudioFileStreamExample demo's http stream, the param inNumberPackets is bigger than 1 at most times; when I play the mms stream, the inNumberPackets is always 1. audioQueue only play the first packet, all of the last packets are missed.
The log of using AudioFileStreamExample demo's http stream:
*AudioStreamBasicDescription info ----*
SampleRate :44100.000000
FormatID :778924083
FormatFlags :0
BytesPerPacket :0
FramesPerPacket :1152
BytesPerFrame :0
ChannelsPerFrame :2
BitsPerChannel :0
got data. bytes: 1253 packets: 2
packetSize:626
kAQBufSize:24576
bytesFilled:0
packetSize:627
kAQBufSize:24576
bytesFilled:626
got data. bytes: 627 packets: 1
packetSize:627
kAQBufSize:24576
bytesFilled:1253
The log of using parsed mms stream:
*AudioStreamBasicDescription info ----*
SampleRate :48000.000000
FormatID :1819304813
FormatFlags :12
BytesPerPacket :4
FramesPerPacket :1
BytesPerFrame :4
ChannelsPerFrame :2
BitsPerChannel :16
got data. bytes: 4498 packets: 1
packetSize:216
kAQBufSize:24576
bytesFilled:0
got data. bytes: 1090 packets: 1
packetSize:576
kAQBufSize:24576
bytesFilled:216
got data. bytes: 3064 packets: 1
packetSize:576
kAQBufSize:24576
bytesFilled:792
got data. bytes: 3978 packets: 1
packetSize:252
kAQBufSize:24576
bytesFilled:1368
I don't know how to solve this problem. It seems that MyPacketsProc function gets wrong params, but how to get fix it?
thanks very much.
Same problem happens with me also,
it seems it happens when AudioQueue callback doesn't invoke on the time.