Is there way to set up an observer / callback on an AVPlayer to get notified when the frame changes?
I am aware of both addBoundaryTimeObserver and addPeriodicTimeObserver however these are approximations that require me to estimate the frame rate, etc.
There is a note that:
General State Observations: You can use Key-value observing (KVO) to observe state changes to many of the player’s dynamic properties, such as its currentItem or its playback rate. You should register and unregister for KVO change notifications on the main thread. This avoids the possibility of receiving a partial notification if a change is being made on another thread. AV Foundation invokes observeValue(forKeyPath:of:change:context:) on the main thread, even if the change operation is made on another thread.
However currentTime on AVPlayerItem is a method, not a property so I cannot use KVO for that.
You could add an AVPlayerItemVideoOutput to your AVPlayerItem and periodically poll the output with hasNewPixelBufferForItemTime which will tell you of the arrival of a new frame. However you then need to acquire the frame with copyPixelBufferForItemTime, so you should probably immediately release it. Here's an example of setting up AVPlayerItemVideoOutput. This is polling, so you could realise late or even miss a frame change.
You could also quickly preprocess the video file (if it is a file) without decompressing the frames, to determine the frame presentation time stamps. You could feed those timestamps one at a time to addBoundaryTimeObserver to decide when you'd crossed a frame boundary. Here's an example of parsing a video file.
AVSampleBufferDisplayLayer, which is a lower level AVPlayerLayer, that lets you feed it video frame CMSampleBuffers looks like a promising way to find out when a frame changes, but it doesn't seem to tell you when it has displayed one of the sample buffers you gave it. And I don't think AVSampleBufferDisplayLayer handles audio, either.
You could also reimplement the AVPlayer playback system - then you'd be painfully (and accurately) aware of frame changes (and audio changes, and opengl/metal). Surely that kind of effort is not required here. What kind of feature are are you trying to implement exactly?
Related
I have a TTrackBar and a TMediaPlayer, I'm looking for a way to change the TTrackBar position according to the TMediaPlayer position using the LiveBindigs feature.
The problem is, there is no event on the TMediaPlayer to watch the changes of the TMediaPlayer.Position property, so my TTrackBar.Position can't synchronize.
Is it possible to watch the changes of a component property without trigger an event?
Not it is not possible to monitor changes of certain property without suitable event.
And you would not want to have any event binded to MediaPlayer.Position property either. Why?
For instance when you are playing a video position is changed for each and every frame which menans that when playing a video with 30 FPS such event would be fired 30 times per second. So depending on the code in that event it could quickly bring your application to a crawl.
So best suggestion that I can give you is for you to place a timer on your form and then check media player position in certain intervals to update your TrackBar. I believe one second interval would is more than enough but you can make it shorter if you will.
Just make sure that if you also use TrackBar for seeking ability to use some control variable to see whether the TrackBar position is being updated by user or by your Timer. Other vise you will end up with weird stuttering (happened to me the first time).
As for achieving all this with LiveBindings alone I don't think it is possible.
I want to implement animation like This link while application interact with server.
If someone has example to implement this kind of animation then please provide me link.
Start animation
Server request in background thread
End animation in completion block of server request in the main thread
If you want that specific one I think you are going to have to make it yourself, but if you don't here are some open source alternatives.
DGActivityIndicatorView
MONActivityIndicatorView
YRActivityIndicator
interaction designer at Creativedash here =]
For something like that you'd make a png sequence, and set the animationImages property of a UIImageView. iOS is great at animating png sequences.
When your load starts, do yourLoadingImage.startAnimating() and when it's over (in the completion for your request), call yourLoadingImage.stopAnimating().
There's a property called animationDuration, and you need to set that as well, just to indicate how long the animation loop is - this doesn't indicate how long the animation will play for. It loops until you tell it to stop.
yourLoadingImage.animationDuration = 1 //set the loop duration to 1s
We actually did a short blog post on this process, and it can be the same for really any custom loader. http://news.ui8.net/create-a-custom-animated-loading-indicator-with-swift/
I know it's sounds silly but just to clear a point.
Is there any chance that view did load will be called before didBecomeActive ?
Is it totally impossible ?
EDIT
We have a crash that happens when user is coming back to the app from the background and we start to use openGL. The crash error points that we try to use openGL in the background.
It is important to say that our app lives in the background as a VOIP app.
We try to figure out if there is a chance that somehow we are triggering something in the background thats causes the app restart openGl in the background.
In the stack we see:
[VideoCallViewController viewDidLoad] (VideoCallViewController.m:283)
And few lines after that:
[GPUImageContext createContext]
And finally:
gpus_ReturnNotPermittedKillClient + 10
We are trying to figure out if there is a way that [VideoCallViewController viewDidLoad] was called in the background or that we must assume that we are in the foreground, and somehow moving to the background right after the viewDidLoad ?
Second option
The second option is that we are indeed moving to the background right after the viewDidLoad. The point here is that we are listening to AppWillResignActive and we pause the GPUIMage. So we can not understand why do we get the crash ?
Thanks
Thanks
When / Where do you instantiate the various GPUImage objects that you are using? Is it within viewDidLoad or possibly within init:?
It's just pure rampant speculation here since you didn't really post any code...
but if you are disposing of objects when the app heads to the background that are not then re-created when it comes back to the foreground (perhaps because the viewController was retained by a parent, and therefore init: was not called again but viewDidLoad was...) Then you may be trying to send OpenGL messages to objects that don't actually exist anymore.
On the (probably unlikely) chance that my speculation is right, you could easily fix it with the common "getter" pattern of:
- (GPUImageObjectOfInterest*)instanceOfObject {
if (!_classVariableOfThisType) {
_classVariableOfThisType = [[GPUImageObjectOfInterest alloc] init];
// custom configuration, etc...
}
return _classVariableOfThisType;
}
and then use [self instanceOfObject]; wherever you used to use _classVariableOfThisType
It's a low overhead, but reasonably foolproof way of making sure a key object exists under a wide range of app interruption / background & foreground & low memory conditions.
Don't be shy to post too much code though, we can read through an entire class if needed. Some of us like reading code! (and it will really help the quality of response you get...)
While playing a video, I'm seeing rate change notifications from AVPlayer that don't seem to be connected to app activity.
When my app receives a UIApplicationDidEnterBackgroundNotification notification, I tell the AVPlayer to pause. The logic is that it should come back to the foreground at the same place the user left. If I do not call pause when going to the background, the problem doesn't appear.
The sequence of events sent to the player is pause, seekToTime:, play. Generally, this works fine but, after the app has been sent to the background and then returned to the foreground, each play invocation results in two rate changes from the AVPlayer. The first is to 1 and the second, immediately following, is to 0. This pattern continues for each call to -[AVPlayer play] as long as that player instance is in use.
I'm able to put a breakpoint on -[AVPlayer pause] and I do not see it being hit when the rate changes to 0. If I comment out the seekToTime: call, the problem goes away. If I use seekToTime:completionHandler:, I also get the same problem although my block's finished parameter is YES.
Apart from "how do I fix this", I'm interested in any details about how to detect the reason for rate changes in AVPlayer that aren't connected to play/pause. (Putting a breakpoint on -[AVPlayer setRate:] never seems to trigger.)
(One workaround that almost works is to save the player position when entering the background, let it play, and fix the position when returning to the foreground. This also requires some manipulation of audio levels, which is probably doable, but another problem is that not all background notifications indicate that the view has been obscured (e.g. double-tap home button). This leads to cases where my workaround shows a distracting moving image when the app is interrupted but still visible.)
Suggestions?
(A last bit of extra information: In all the cases I've tried, I eventually get to a state where the AVPlayer is changing the rate from 1 to 0 moments after I invoke 'play' if I've returned from the background and then performed a seek. There are things I can do to make it less frequent but none that eliminate it except getting rid of the AVPlayer and creating a new instance. This results is very long delays but is better than a complete malfunction...I guess.
I have some evidence that the seek distance affects the result, which suggests that the error might be in the underlying buffering mechanism. Without knowing what causes the rate change (other than play/pause) I don't see a way to investigate further.)
You are probably getting an AVPlayerItemPlaybackStalledNotification.
Try this:
[[NSNotificationCenter defaultCenter]
addObserverForName:AVPlayerItemPlaybackStalledNotification
object:cell.avPlayerItem
queue:[NSOperationQueue mainQueue]
usingBlock:^(NSNotification *note) {
DDLogVerbose(#"%#", #"AVPlayerItemPlaybackStalledNotification");
}];
However, Apple docs say Playback will continue once a sufficient amount of media has subsequently been delivered. (https://developer.apple.com/library/iOS/documentation/AVFoundation/Reference/AVPlayerItem_Class/Reference/Reference.html#//apple_ref/doc/uid/TP40009532-CH1-SW83)
This is not happening for me or you.
I am still tracking the reason down.
EDIT:
This was the problem for me. When AVPlayerItem.likelyToKeepUp was YES then it wouldn't happen.
Not sure why it wasn't resuming.
EDIT:
From the Apple docs there is a situation where playback will not resume:
This property communicates a prediction of playability. Factors
considered in this prediction include I/O throughput and media decode
performance. It is possible for playbackLikelyToKeepUp to indicate NO
while the property playbackBufferFull indicates YES. In this event the
playback buffer has reached capacity but there isn't the statistical
data to support a prediction that playback is likely to keep up in the
future.
It is up to you to decide whether to continue media playback.
In order to invoke some events for the specified time, for example, when the video goes to 10.0s, or 20.0s , there should be some events to be invoked, is that possible to observe some property of the instance of MPMoviePlayerController for such cases ? Or any other solutions ?
I had to do almost the same thing and I didn't find any good solution except by using my own NSTimer and look at some moments the current time of the MPMoviePlayer...
So basically, every seconds I'm watching what current time it is and when I'm interrested in the time, I do my stuff.