Capture UIImage frame of AVPlayer streaming .m3u8 for given time? - ios

Question

So I have an AVPlayer that playing a live stream of an .m3u8 video and from what I've found searching it looks like you can't use AVAssetImageGenerator to make a screen shot but instead should use AVPlayerItemVideoOutput and
- (CVPixelBufferRef)copyPixelBufferForItemTime:(CMTime)itemTime itemTimeForDisplay:(CMTime *)outItemTimeForDisplay
but when I try and get the outputs from my AVPlayer.
NSArray *outputs = self.mainPlayer.currentItem.outputs;
I get an empty array.
The video plays just fine. Ultimatly what I want is a method like this.
-(UIImage *)frameFor:(CMTime)time;
At some point the CALayer on the view needs to be getting this image data so their has to be a way to grab that at some point. I tried just capturing the CALayer my AVPLayerLayer is attached to but I don't get anything more than the blank view color (bright pink just to make sure its return something). Their has to be some way of grabbing this data.

Answer 1

You're getting an empty array when you do
NSArray *outputs = self.mainPlayer.currentItem.outputs;
because the AVPlayerItemVideoOutput object needs to be added first
NSDictionary *settings = #{(id)kCVPixelBufferPixelFormatTypeKey : [NSNumber numberWithInt:kCVPixelFormatType_32BGRA]};
self.videoOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:settings];
[playerItem addOutput:self.videoOutput];
and then when the AVPlayerItem status is AVPlayerStatusReadyToPlay you can capture the current frame using copyPixelBufferForItemTime:itemTimeForDisplay. For that, you need to create your AVPlayer object and add an observer to the AVPlayerItem status property
self.player = [AVPlayer playerWithPlayerItem:playerItem];
[self.player.currentItem addObserver:self forKeyPath:#"status" options:0 context:NULL];
and then in your callback function
- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
{
if (self.player.currentItem.status == AVPlayerStatusReadyToPlay)
{
CMTime currentTime = self.player.currentItem.currentTime;
CVPixelBufferRef buffer = [self.videoOutput copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil];
CIImage *ciImage = [CIImage imageWithCVPixelBuffer:buffer];
UIImage *image = [UIImage imageWithCIImage:ciImage];
// use `image`
}
}
If you only need to capture a thumbnail and you don't have to play the video first, you can use a simple utility that I've created that follows this logic.
(https://github.com/acotilla91/ACThumbnailGenerator)
How to use:
double bitRate = 1000000; // force video bit rate (can be use to cap video quality and improve performance). Pass 0 to use default bit rate.
self.thumbnailGenerator = [[ACThumbnailGenerator alloc] initWithPreferredBitRate:bitRate];
NSURL *videoURL = [NSURL URLWithString:#"http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/sl.m3u8"];
int position = 10; // video position (in seconds) from where thumbnail should be extracted. Always pass 0 for live streams.
[self.thumbnailGenerator loadImageFrom:videoURL position:position withCompletionBlock:^(UIImage *image) {
// use `image`
}];
Hope it helps.
Swift version:
ACThumbnailGenerator-Swift
Usage:
var generator: ACThumbnailGenerator!
func captureSomeImage() {
let streamUrl = URL(string: "https://p-events-delivery.akamaized.net/18oijbasfvuhbfsdvoijhbsdfvljkb6/m3u8/hls_vod_mvp.m3u8")!
generator = ACThumbnailGenerator(streamUrl: streamUrl)
generator.delegate = self
generator.captureImage(at: 300)
}
func generator(_ generator: ACThumbnailGenerator, didCapture image: UIImage, at position: Double) {
// Use `image`
}