My app uses ffmpeg to parse mms streaming to buffers, and uses Audio queue to play the buffered data. Now I can confirm the buffered data is good, but I can not play it. The structure of playing is based on the AudioFileStreamExample demo's client, and it can play the http stream(AudioFileStreamExample demo's server). I notice in the callback function:
void MyPacketsProc(void *inClientData, UInt32 inNumberBytes,
UInt32 inNumberPackets, const void *inInputData,
AudioStreamPacketDescription *inPacketDescriptions),
When I directly play the AudioFileStreamExample demo's http stream, the param inNumberPackets is bigger than 1 at most times; when I play the mms stream, the inNumberPackets is always 1. audioQueue only play the first packet, all of the last packets are missed.
The log of using AudioFileStreamExample demo's http stream:
*AudioStreamBasicDescription info ----*
SampleRate :44100.000000
FormatID :778924083
FormatFlags :0
BytesPerPacket :0
FramesPerPacket :1152
BytesPerFrame :0
ChannelsPerFrame :2
BitsPerChannel :0
got data. bytes: 1253 packets: 2
packetSize:626
kAQBufSize:24576
bytesFilled:0
packetSize:627
kAQBufSize:24576
bytesFilled:626
got data. bytes: 627 packets: 1
packetSize:627
kAQBufSize:24576
bytesFilled:1253
The log of using parsed mms stream:
*AudioStreamBasicDescription info ----*
SampleRate :48000.000000
FormatID :1819304813
FormatFlags :12
BytesPerPacket :4
FramesPerPacket :1
BytesPerFrame :4
ChannelsPerFrame :2
BitsPerChannel :16
got data. bytes: 4498 packets: 1
packetSize:216
kAQBufSize:24576
bytesFilled:0
got data. bytes: 1090 packets: 1
packetSize:576
kAQBufSize:24576
bytesFilled:216
got data. bytes: 3064 packets: 1
packetSize:576
kAQBufSize:24576
bytesFilled:792
got data. bytes: 3978 packets: 1
packetSize:252
kAQBufSize:24576
bytesFilled:1368
I don't know how to solve this problem. It seems that MyPacketsProc function gets wrong params, but how to get fix it?
thanks very much.
Same problem happens with me also,
it seems it happens when AudioQueue callback doesn't invoke on the time.
Related
Please guide me to resolve this issue.
I have parsed the h264 video stream and identified the frames[I/P/B]. I have followed the below steps.
• NAL Units start code: 00 00 01 X Y
• X = IDR Picture NAL Units (25, 45, 65)
• Y = Non IDR Picture NAL Units (01, 21, 41, 61) ; 01 = b-frames, 41 = p-frames
Now my question is how to know the length of individual frames so that i can write each frames to a file. Please give some help.
Regards,
Spk
Ok, so your source is an annex-b formated elementary stream. Basically every NALu begins with a start code (2 or more 0x00 bytes followed by a 0x01 byte). The next byte contains the type (the first 5 bits). The rest is payload. The NALU ends when the next start code in encountered, or you reach the end of the stream. So, to get the length, you must look for the next start code and subtract.
You will likely find this post useful. Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream
i am using AudioQueues to get Chunks of audio samples.
here is my callback method
void AQRecorder::MyInputBufferHandler( void * inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp * inStartTime,
UInt32 inNumPackets,
const AudioStreamPacketDescription* inPacketDesc)
there is api which expect me to send byte array (that i am not familiar with) which variable should i send in this case?
there is not a lot of docs about this one
The mDataByteSize element of the C struct pointed to by inPacketDesc will tell you the number of bytes per packet. And the inNumPackets function parameter is the number of packets sent to your Audio Queue callback function. Multiply the two to get the total number of bytes to send.
The app might also have set up the number of bytes per packet when configuring the Audio Queue, so you could just use that number.
I've specified and instantiated two Audio Units: a multichannel mixer unit and a generator of subtype AudioFilePlayer.
I would have thought I needed to set the ASBD of the filePlayer's output to match the ASBD I set for the mixer input. However when I attempt to set the filePlayer's output I get a kAudioUnitErr_FormatNotSupported (-10868) error.
Here's the stream format I set on the mixer input (successfully) and am also trying to set on the filePlayer (it's the monostream format copied from Apple's mixerhost sample project):
Sample Rate: 44100
Format ID: lpcm
Format Flags: C
Bytes per Packet: 2
Frames per Packet: 1
Bytes per Frame: 2
Channels per Frame: 1
Bits per Channel: 16
In the course of troubleshooting this I queried the filePlayer AU for the format it is 'natively' set to. This is what's returned:
Sample Rate: 44100
Format ID: lpcm
Format Flags: 29
Bytes per Packet: 4
Frames per Packet: 1
Bytes per Frame: 4
Channels per Frame: 2
Bits per Channel: 32
All the example code I've found sends the output of the filePlayer unit to an effect unit and set the filePlayer's output to match the ASBD set for the effect unit. Given I have no effect unit it seems like setting the filePlayer's output to the mixer input's ASBD would be the correct - and required - thing to do.
How have you configured the AUGraph? I might need to see some code to help you out.
Setting the output scope of AUMultiChannelMixer ASBD once only (as in MixerHost) works. However if you have any kind of effect at all, you will need to think about where their ASBDs are defined and how you arrange your code so CoreAudio does not jump in and mess with your effects AudioUnits ASBDs. By messing with I mean overriding your ASBD to the default kAudioFormatFlagIsFloat, kAudioFormatFlagIsPacked, 2 channels, non-interleaved. This was a big pain for me at first.
I would set the effects AudioUnits to their default ASBD. Assuming you have connected the AUFilePlayer node, then you can pull it out later in the program like this
result = AUGraphNodeInfo (processingGraph,
filePlayerNode,
NULL,
&filePlayerUnit);
And then proceed to set
AudioUnitSetProperty(filePlayerUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
0,
&monoStreamFormat,
sizeof(monoStreamFormat));
Hopefully this helps.
Basically I didn't bother setting the filePlayer ASBD but rather retrieved the 'native' ASBD it was set to and updated only the sample rate and channel count.
Likewise I didn't set input on the mixer and let the mixer figure it's format out.
I need help understanding the following ASBD. It's the default ASBD assigned to a fresh instance of RemoteIO (I got it by executing AudioUnitGetProperty(..., kAudioUnitProperty_StreamFormat, ...) on the RemoteIO audio unit, right after allocating and initializing it).
Float64 mSampleRate 44100
UInt32 mFormatID 1819304813
UInt32 mFormatFlags 41
UInt32 mBytesPerPacket 4
UInt32 mFramesPerPacket 1
UInt32 mBytesPerFrame 4
UInt32 mChannelsPerFrame 2
UInt32 mBitsPerChannel 32
UInt32 mReserved 0
The question is, shouldn't mBytesPerFrame be 8? If I have 32 bits (4 bytes) per channel, and 2 channels per frame, shouldn't each frame be 8 bytes long (instead of 4)?
Thanks in advance.
The value of mBytesPerFrame depends on mFormatFlags. From CoreAudioTypes.h:
Typically, when an ASBD is being used, the fields describe the complete layout
of the sample data in the buffers that are represented by this description -
where typically those buffers are represented by an AudioBuffer that is
contained in an AudioBufferList.
However, when an ASBD has the kAudioFormatFlagIsNonInterleaved flag, the
AudioBufferList has a different structure and semantic. In this case, the ASBD
fields will describe the format of ONE of the AudioBuffers that are contained in
the list, AND each AudioBuffer in the list is determined to have a single (mono)
channel of audio data. Then, the ASBD's mChannelsPerFrame will indicate the
total number of AudioBuffers that are contained within the AudioBufferList -
where each buffer contains one channel. This is used primarily with the
AudioUnit (and AudioConverter) representation of this list - and won't be found
in the AudioHardware usage of this structure.
I believe that because the format flags specify kAudioFormatFlagIsNonInterleaved it follows that the size of a frame in any buffer can only be the size of a 1 channel frame. If this is correct mChannelsPerFrame is certainly a confusing name.
I hope someone else will confirm / clarify this.
I need to parse ffmpegs meta data output but it is different for some reason between a wav and a flac file.
Flac:
(int) 14 => ' Duration: 00:03:18.93, bitrate: 1045 kb/s',
(int) 15 => ' Stream #0:0: Audio: flac, 44100 Hz, stereo, s16',
Wav:
(int) 13 => ' Duration: 00:00:15.00, bitrate: 1411 kb/s',
(int) 14 => ' Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s',
I could get the bitrate from the duration line too I think but why is it different? And will there be more differences in future releases? It really sucks that there is no better way to get the information from ffmpeg than to parse it's output. Any better ideas?
Here is my whole ffmpeg output and my parsed result:
http://pastebin.com/4qJfzZNL
I solved it finally by using ffprobe which comes with ffmpeg.
ffprobe -v quiet -show_streams -show_format -show_error -print_format <format> <file>
See the writers section of the documentation about the formats it supports, I've used json but xml, csv and ini are also supported.
The stream line provides different information because each codec has different parameters. You will need to parse the line and depending on the audio type you will need to understand those parameters that come after it.
You could just use the bitrate in the duration line, but this may be misleading without knowledge of which codec is in use.