I'm trying to determine the best way to tell if an AVPlayerItem is buffered to the end of the stream. Not just that the buffer is full, but that the buffer contains everything needed to play the rest of the item without additional buffering. AVPlayerItem offers a is isPlaybackBufferFull call, but that doesn't tell me if any additional buffering needs to happen before the item is finished playing.
My current plan is to combine that with preferredForwardBufferDuration to check if the item will ever need to buffer more, but is this the best way?
For instance:
- (void)observeValueForKeyPath:(NSString*)aKeyPath ofObject:(id)aObject change:(NSDictionary*)aChange context:(void*)aContext
{
if( [aKeyPath isEqualToString:#"playbackBufferFull"] )
{
CMTime theBufferTime = CMTimeMakeWithSeconds( self.currentItem.preferredForwardBufferDuration, 1 );
CMTime theEndBufferTime = CMTimeAdd( self.currentItem.currentTime, theBufferTime );
if( CMTimeCompare( theEndBufferTime, self.currentItem.duration ) >= 0 )
{
// Buffered to the end
}
}
}
I found a pretty good solution to this problem that can be seen below. The proposed solution written in the question didn't really work well because the preferredForwardBufferDuration is set to 0 by default which pretty much makes the solution not viable.
The following the code works pretty well. I call it every second on a timer.
auto theLoadedRanges = self.currentItem.loadedTimeRanges;
CMTime theTotalBufferedDuration = kCMTimeZero;
for( NSValue* theRangeValue in theLoadedRanges )
{
auto theRange = [theRangeValue CMTimeRangeValue];
theTotalBufferedDuration = CMTimeAdd( theTotalBufferedDuration, theRange.duration );
}
auto theDuration = CMTimeGetSeconds( self.currentItem.duration );
if( theDuration > 0 )
{
float thePercent = CMTimeGetSeconds( theTotalBufferedDuration ) / theDuration;
if( thePercent >= 0.99f )
{
// Fully buffered
}
}
I have been playing with the RtspPlay1 from mooncatventures to try and stream a live stream from an ffmpeg streaming source with as little delay as possible. The problem is even when I modify the code to indicate the -analyzedelay 0 flags in RtspPlay1 it does not seem to do anything. I came to this conclusion because the delay is the same on my computer without the -analyzeduration 0 flag as the iOS device. Any thoughts would be helpful.
Here is the command I am trying to emulate on the iPhone:
ffplay rtp:///224.1.1.1:11326 -analyzeduration 0
Here is the modified code I tried with RtspPlay1:
forward_argc=1;
forward_argv[1] = "-analyzeduration";
forward_argv[2] = "0";
//forward_argv[3] = "30";
//forward_argv[4] = "-fast";
//forward_argv[5] = "-sync";
//forward_argv[6] = "video";
//forward_argv[7] = "-drp";
//forward_argv[8] = "-skipidct";
//forward_argv[9] = "10";
//forward_argv[10] = "-skiploop";
//forward_argv[11] = "50";
//forward_argv[12] = "-threads";
//forward_argv[13] = "5";
//argv[14] = "-an";
forward_argv[3] = cString;
NSLog(#"glflag %#\n ",[parms objectForKey:#"glflag"] );
if ([parms objectForKey:#"glflag"]!=#"1") {
forward_argv[4]="0";
}else {
forward_argv[4]="1";
}
forward_argc += 4;
I am using the multiple-output-device feature provided by paMME host API to output audio through multiple stereo devices. I also need to use a single multichannel input device using MME.
- When I configure just the output device and play internally generated audio, there is no problem.
- However problem starts to occur when I configure both the input device and the mulitple-stereo output devices. The application crashes when I try to use more than two channels on the output. That is, if I try to increment the 'out' pointer for more than 2*frames_per_buffer , it crashes, which indicates that buffer has been allocated only to two output channels.
Can anybody throw some light on what could be the problem. The configuration code is given below:
outputParameters.device = paUseHostApiSpecificDeviceSpecification;
outputParameters.channelCount = 8;
outputParameters.sampleFormat = paInt16;
outputParameters.hostApiSpecificStreamInfo = NULL;
wmmeStreamInfo.size = sizeof(PaWinMmeStreamInfo);
wmmeStreamInfo.hostApiType = paMME;
wmmeStreamInfo.version = 1;
wmmeStreamInfo.flags = paWinMmeUseMultipleDevices;
wmmeDeviceAndNumChannels[0].device = selectedDeviceIndex[0];
wmmeDeviceAndNumChannels[0].channelCount = 2;
wmmeDeviceAndNumChannels[1].device = selectedDeviceIndex[1];
wmmeDeviceAndNumChannels[1].channelCount = 2;
wmmeDeviceAndNumChannels[2].device = selectedDeviceIndex[2];
wmmeDeviceAndNumChannels[2].channelCount = 2;
wmmeDeviceAndNumChannels[3].device = selectedDeviceIndex[3];
wmmeDeviceAndNumChannels[3].channelCount = 2;
wmmeStreamInfo.devices = wmmeDeviceAndNumChannels;
wmmeStreamInfo.deviceCount = 4;
outputParameters.suggestedLatency = Pa_GetDeviceInfo( selectedDeviceIndex[0] )->defaultLowOutputLatency;
outputParameters.hostApiSpecificStreamInfo = &wmmeStreamInfo;
inputParameters.device = selectedInputDeviceIndex; /* default output device */
inputParameters.channelCount = 8; /* stereo output */
inputParameters.sampleFormat = paInt16; /* 32 bit floating point output */
inputParameters.suggestedLatency = Pa_GetDeviceInfo( inputParameters.device )->defaultLowInputLatency;
inputParameters.hostApiSpecificStreamInfo = NULL;
Thanks and regards,
Siddharth Kumar.
I have modified the code provided by Tim Boldstad http://timbolstad.com/2010/03/16/core-audio-getting-started-pt2/ (may God bless him), and added a small slider to be able to change the output tone frequency form 40hz to 200000 hz. I now want to be able to use a LPF on the tone generated.
First of all, does any1 have a detailed guide which explains how to do this. I've tried simply adding a node in between, but it doesn't work, apparently, I need to convert 16 bit integer samples to the floating 8.24 format, before giving audio sample inputs to the filter, and then i have to convert it back to 16 bit integer. Is this the problem? or have i connected the node wrongly?
Where am i supposed to set the filters cutoff frequency and other parameters?
Can anyone explain what AudioUnitGetProperty does? Apple documentation on these topics are EXTREMELY fragmented and utterly worthless :(
-(void) initializeAUGraph
{
OSStatus result= noErr;
result = NewAUGraph(&mGraph);
AUNode outputNode;
AUNode mixerNode;
AUNode effectsNode;
AudioComponentDescription effects_desc;
effects_desc.componentType = kAudioUnitType_Effect;
effects_desc.componentSubType = kAudioUnitSubType_LowPassFilter;
effects_desc.componentFlags = 0;
effects_desc.componentFlagsMask = 0;
effects_desc.componentManufacturer = kAudioUnitManufacturer_Apple;
AudioComponentDescription mixer_desc;
mixer_desc.componentType=kAudioUnitType_Mixer;
mixer_desc.componentSubType=kAudioUnitSubType_MultiChannelMixer;
mixer_desc.componentFlags=0;
mixer_desc.componentFlagsMask=0;
mixer_desc.componentManufacturer=kAudioUnitManufacturer_Apple;
AudioComponentDescription output_desc;
output_desc.componentType = kAudioUnitType_Output;
output_desc.componentSubType = kAudioUnitSubType_RemoteIO;
output_desc.componentFlags = 0;
output_desc.componentFlagsMask = 0;
output_desc.componentManufacturer = kAudioUnitManufacturer_Apple;
result= AUGraphAddNode(mGraph, &output_desc, &outputNode);
result= AUGraphAddNode(mGraph, &mixer_desc, &mixerNode);
result=AUGraphAddNode(mGraph, &effects_desc, &effectsNode);
result=AUGraphConnectNodeInput(mGraph, mixerNode, 0, effectsNode, 0);
result=AUGraphConnectNodeInput(mGraph, effectsNode, 0, outputNode, 0);
result=AUGraphOpen(mGraph);
//getting mixxer
result = AUGraphNodeInfo(mGraph, mixerNode, NULL, &mMixer);
result = AUGraphNodeInfo(mGraph, effectsNode, NULL, &mEffects);
UInt32 numbuses = 1;
UInt32 size = sizeof(numbuses);
result = AudioUnitSetProperty(mMixer, kAudioUnitProperty_ElementCount, kAudioUnitScope_Input, 0, &numbuses, size);
//=====
CAStreamBasicDescription desc;
// Loop through and setup a callback for each source you want to send to the mixer.
// Right now we are only doing a single bus so we could do without the loop.
for (int i = 0; i < numbuses; ++i)
{
// Setup render callback struct
// This struct describes the function that will be called
// to provide a buffer of audio samples for the mixer unit.
AURenderCallbackStruct renderCallbackStruct;
renderCallbackStruct.inputProc = &renderInput;
renderCallbackStruct.inputProcRefCon = self;
// Set a callback for the specified node's specified input
result = AUGraphSetNodeInputCallback(mGraph, mixerNode, i, &renderCallbackStruct);
// Get a CAStreamBasicDescription from the mixer bus.
size = sizeof(desc);
result = AudioUnitGetProperty( mMixer,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
i,
&desc,
&size);
// Initializes the structure to 0 to ensure there are no spurious values.
memset (&desc, 0, sizeof (desc));
// Make modifications to the CAStreamBasicDescription
// We're going to use 16 bit Signed Ints because they're easier to deal with
// The Mixer unit will accept either 16 bit signed integers or
// 32 bit 8.24 fixed point integers.
desc.mSampleRate = kGraphSampleRate; // set sample rate
desc.mFormatID = kAudioFormatLinearPCM;
desc.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
desc.mBitsPerChannel = sizeof(AudioSampleType) * 8; // AudioSampleType == 16 bit signed ints
desc.mChannelsPerFrame = 1;
desc.mFramesPerPacket = 1;
desc.mBytesPerFrame = ( desc.mBitsPerChannel / 8 ) * desc.mChannelsPerFrame;
desc.mBytesPerPacket = desc.mBytesPerFrame * desc.mFramesPerPacket;
printf("Mixer file format: "); desc.Print();
// Apply the modified CAStreamBasicDescription to the mixer input bus
result = AudioUnitSetProperty( mMixer,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
i,
&desc,
sizeof(desc));
}
// Apply the CAStreamBasicDescription to the mixer output bus
result = AudioUnitSetProperty( mMixer,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
0,
&desc,
sizeof(desc));
//************************************************************
//*** Setup the audio output stream ***
//************************************************************
// Get a CAStreamBasicDescription from the output Audio Unit
result = AudioUnitGetProperty( mMixer,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
0,
&desc,
&size);
// Initializes the structure to 0 to ensure there are no spurious values.
memset (&desc, 0, sizeof (desc));
// Make modifications to the CAStreamBasicDescription
// AUCanonical on the iPhone is the 8.24 integer format that is native to the iPhone.
// The Mixer unit does the format shifting for you.
desc.SetAUCanonical(1, true);
desc.mSampleRate = kGraphSampleRate;
// Apply the modified CAStreamBasicDescription to the output Audio Unit
result = AudioUnitSetProperty( mMixer,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
0,
&desc,
sizeof(desc));
// Once everything is set up call initialize to validate connections
result = AUGraphInitialize(mGraph);
}
Can anyone explain what AudioUnitGetProperty does?
Well, it gets the value of a property from an Audio Unit. A "property" is typically something you deal with as a programmer (e.g. audio stream format, connection state), whereas a "parameter" is usually something you expose to the user (e.g. low pass cutoff frequency, mixer volume). Notice that there are AudioUnitGetParameter and AudioUnitSetParameter functions to compliment the AudioUnitGetProperty and AudioUnitSetProperty functions.
You're basically expected to "just know" what an Audio Unit's properties / parameters are and what values they're expecting. The best source of documentation on this are two headers in AudioUnit.framework. Namely, AudioUnitProperties.h and AudioUnitParameters.h. The next best source is Xcode's autocomplete. For example, the AULowPass' parameters are kLowPassParam_CutoffFrequency and kLowPassParam_Resonance, so you can just type kLowPassParam and Xcode will show you what's available. The other AUs typically follow this scheme.
...but it doesn't work, apparently
I'm going to need more information. Do you mean you just can't hear the difference? The AULowPass starts with a very high cutoff frequency, so unless you set it something lower you probably won't hear any difference at all.
Try setting the cutoff frequency to something quite low, for example 500hz. You do that like this:
AudioUnitSetParameter(mEffects,
kLowPassParam_CutoffFrequency,
kAudioUnitScope_Global,
0,
500,
0);
I want to export every frame in a *.mov-Movie-File, so I do this:
GoToBeginningOfMovie(movie);
TimeValue startPoint = 0;
long gnitFrames = 0;
while (startPoint >= 0) {
GetMovieNextInterestingTime(movie, nextTimeStep, 0, &whichMediaType, startPoint, 0, &startPoint, NULL);
gnitFrames++;
}
the problem is, the count of gnitFrames is different (many more) than when I call this:
Track track = GetMovieIndTrack(movie, 1);
Media media = GetTrackMedia(track);
OSType mediatype;
MediaHandler mediahandler = GetMediaHandler(media);
GetMediaHandlerDescription(media, &mediatype, nil, nil);
MediaGetName(mediahandler, medianame, 0, nil);
long nsamples = GetMediaSampleCount(media);
nsamples gives me the correct frame-count. So now my question: how can I do that to get to every frame in a movie just once? (When I export the frame now after I called GetNextInterestingTime, a frame is exported multiple times, sometimes even 25 times)
My operating system is Windows XP.
Using nextTimeStep might be problematic as a timestep does not necessarily have to match a (video) media sample causing GetMovieNextInterestingTime() to return superfluous time stamps.
If all you want to do is to count / locate all frames in a video media, try using nextTimeMediaSample along with GetMediaNextInterestingDisplayTime() on the video Media like this:
...
TimeValue64 start = 0;
TimeValue64 sample_time = 0;
TimeValue64 sample_duration = -1;
int frames = 0;
while( sample_time != -1 ) {
GetMediaNextInterestingDisplayTime( media, nextTimeMediaSample | nextTimeEdgeOK, start, fixed1, &sample_time, &sample_duration );
if( sample_time != -1 ) {
++frames;
}
...
start += sample_duration;
}
...
Caveat:
According to the Q&A article below this approach is not supposed to work out for f.e. MPEG
but for many other formats it works like a charm in my experience.
Technical Q&A QTMTB54: How do I count the frames in an MPEG movie?