ios UICollectionView slow scrolling and memory warning - ios

I am currently working on a project that involves a UICollectionView populated by AVAssets imported from a UIImagepickerController, after 10 or so item are in the collection, Scrolling becomes laggy and slow, and occasionally I receive memory warnings. I believe the problem to be in the thumbnail generation which happens in realtime, here is the code i use:
- (void) setAsset:(AVAsset *)asset
{
_asset = asset;
AVAssetImageGenerator *generate = [[AVAssetImageGenerator alloc] initWithAsset:_asset];
NSError *err = NULL;
CMTime time = CMTimeMake(1, 60);
generate.appliesPreferredTrackTransform = YES;
CGImageRef imgRef = [generate copyCGImageAtTime:time actualTime:NULL error:&err];
self.VideoImageView.image = [UIImage imageWithCGImage:(imgRef)];
}
Is there another less "expensive way" to achieve this without delay?
Any help on the matter would be greatly appreciated.

You can do the real time thumbnail generation on a background thread, and then jump back to the main thread when the operation is done to set the actual thumbnail.
Right now you're doing everything on the main thread, which blocks the UI, and makes the scrolling jerky.

You can set a placeholder image in your cell and generate thumbnail in background queue.
Then set it to your image view.
AsynImageView might be of some use to you.

Open another thread to deal with the image thing. When your image done switch back to main thread to update a cell. You can do this with GCD

Here are the code that asynchronous ganerate thumbnail image for video.
-(void)generateImage
{
NSURL *url = [NSURL fileURLWithPath:_videoPath];
AVURLAsset *asset=[[AVURLAsset alloc] initWithURL:url options:nil];
AVAssetImageGenerator *generator = [[AVAssetImageGenerator alloc] initWithAsset:asset];
generator.appliesPreferredTrackTransform=TRUE;
CMTime thumbTime = CMTimeMakeWithSeconds(30,30);
AVAssetImageGeneratorCompletionHandler handler = ^(CMTime requestedTime, CGImageRef im, CMTime actualTime, AVAssetImageGeneratorResult result, NSError *error){
if (result != AVAssetImageGeneratorSucceeded) {
NSLog(#"couldn't generate thumbnail, error:%#", error);
}
// TODO Do something with the image
};
CGSize maxSize = CGSizeMake(128, 128);
generator.maximumSize = maxSize;
[generator generateCGImagesAsynchronouslyForTimes:[NSArray arrayWithObject:[NSValue valueWithCMTime:thumbTime]] completionHandler:handler];
}
Hope this help you.

Related

AVAssetImageGenerator fails at copying image

I am using AVAssetImageGenerator to create an image from the last frame of a video. This usually works fine, but every now and then copyCGImageAtTime fails with the error
NSLocalizedDescription = "Cannot Open";
NSLocalizedFailureReason = "This media cannot be used.";
NSUnderlyingError = "Error Domain=NSOSStatusErrorDomain Code=-12431";
I am verifying that the AVAsset is not nil and I'm pulling the CMTime directly from the asset, so I do not understand why this keeps happening. This only happens when trying to get the last frame, if I use kCMTimeZero instead, it seems to work.
- (void)getLastFrameFromAsset:(AVAsset *)asset completionHandler:(void (^)(UIImage *image))completion
{
NSAssert(asset, #"Tried to generate last frame from nil asset");
AVAssetImageGenerator *gen = [[AVAssetImageGenerator alloc] initWithAsset:asset];
gen.requestedTimeToleranceBefore = kCMTimeZero;
gen.requestedTimeToleranceAfter = kCMTimeZero;
gen.appliesPreferredTrackTransform = YES;
CMTime time = [asset duration];
NSError *error = nil;
CMTime actualTime;
CGImageRef imageRef = [gen copyCGImageAtTime:time actualTime:&actualTime error:&error];
UIImage *image = [[UIImage alloc] initWithCGImage:imageRef];
NSAssert(image, #"Failed at generating image from asset's last frame");
completion(image);
CGImageRelease(imageRef);
}
This seems to be related, but it did not solve my problem.
Nothing guarantees that your asset's video track exists at [asset duration]. It's duration can be shorter than the whole asset. Since you set the tolerance to kCMTimeZero the only possible resolution is failure.
Edit: To clarify, the issue emerges when you have an asset with audio track slightly longer than the video track.

Getting thumbnail of a video picked from photos library

I know this question has been asked many times on stackoverflow. But my problem is different.
I am iterating on the albumns of photos library to get all videos and their thumbnails.
Now, the problem is, my code is very slow to get the thumbnail of each video. For example, there is 14 videos in my camera roll and the total time taken to generate the thumbnail is around 3-4 seconds. I am using this code.
+(UIImage*)imageFromVideoAtURL:(NSURL*)contentURL {
UIImage* theImage = nil;
AVURLAsset* asset = [[AVURLAsset alloc] initWithURL:contentURL options:nil];
AVAssetImageGenerator* generator = [[AVAssetImageGenerator alloc] initWithAsset:asset];
generator.appliesPreferredTrackTransform = YES;
NSError* err = NULL;
CMTime time = CMTimeMake(1, 60);
CGImageRef imgRef = [generator copyCGImageAtTime:time actualTime:NULL error:&err];
theImage = [[[UIImage alloc] initWithCGImage:imgRef] autorelease];
CGImageRelease(imgRef);
[asset release];
[generator release];
return theImage;
}
I am finding a way to get the thumbnail of all videos very fast so that user has not to wait. I have seen apps on Apple store that are doing the same thing in just micro seconds. Please help.
I have also tried this code to generate the thumbnail, it is also very slow.
MPMoviePlayerController *moviePlayer = [[MPMoviePlayerController alloc] initWithContentURL:mediaUrl];
moviePlayer.shouldAutoplay = NO;
UIImage *thumbnail = [[moviePlayer thumbnailImageAtTime:0.0 timeOption:MPMovieTimeOptionNearestKeyFrame] retain];
[imageView setImage:thumbnail]; //imageView is a UIImageView
If you are loading videos from image library, it should already have the embedded thumbnail of the video. This can be accessed using thumbnail or aspectRatioThumnail methods of ALAsset class.
So in your case the thumbnails could be loaded like:
ALAssetLibrary* lib = [ALAssetLibrary new];
[lib assetForURL:contentURL resultBlock:^(ALAsset* asset) {
CGImageRef thumb = [asset thumbnail];
dispatch_async(dispatch_get_main_queue(), ^{
//do any UI operation here with thumb
});
}];
Please make sure to make any UIKit call in the main queue as assetForURL:: method may invoke the resultBlock in some background thread.

Generating UIImage from GPUImage video frame

I'm trying to generate a UIImage from a video frame captured by GPUImage. I've done a lot of AVFoundation video work, but i'm new to using GPUImage. I've subclassed GPUImageVideoCamera and added this method, but the UIImage is always nil. If anyone can tell me where i've gone so horribly wrong, i'd be very appreciative!
- (void)processVideoSampleBuffer:( CMSampleBufferRef )sampleBuffer
{
[super processVideoSampleBuffer:sampleBuffer]; // to let GPUImage do it's processing first
if (!self.thumbnailGenerated)
{
CMTime timestamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
NSLog(#"%f", (float)timestamp.value / timestamp.timescale);
self.thumbnailGenerated = YES;
dispatch_sync(dispatch_get_main_queue(), ^
{
// generate a preview frame from the last filter in the camera filter chain
UIImage *thumbnailImage = [UIImage imageWithCGImage:[[self.targets lastObject] newCGImageFromCurrentlyProcessedOutput]];
NSString *pathToMovie = [NSHomeDirectory() stringByAppendingPathComponent:#"Documents/Thumbnail.png"];
[UIImagePNGRepresentation(thumbnailImage) writeToFile:pathToMovie atomically:YES];
});
}
}
I've used this code to generate a CGImageRef of the first frame, used for thumbnail
AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:videoURl options:nil];
AVAssetImageGenerator *imageGenerator = [AVAssetImageGenerator assetImageGeneratorWithAsset:asset];
[imageGenerator setAppliesPreferredTrackTransform:YES];
NSData *videoData = [NSData dataWithContentsOfURL:asset.URL];
CGImageRef image = [imageGenerator copyCGImageAtTime:kCMTimeZero actualTime:nil error:&error];
You can replace kCMTimeZero with some actual value to get the frame you'd like.
After that, you'll have to convert your CGImageRef to a UIImage.
I'm not sure if that is of any help, but I'm getting thumbnail while processing video. For this purpose I'm using
videoInput --> someMyOperations --> fileOutput
someMyOperations --> imageOutput //imageOutput is PictureOutput()
videoInput.start() //that needs to be called!
imageOutput.saveNextFrameToUrl(coverUrl, format: .jpg) { file in
//here goes the code what to do with thumbnail
videoInput.cancel() //quite probably here you want this
}
That's guessing - I don't see your code, but for me this works.

ios - generateCGImagesAsynchronouslyForTimes taking too long

The problem I have is loading 20 images from video takes too long. The more thumbnails I want to get, the longer I have to wait. Method I use is generateCGImagesAsynchronouslyForTimes. Does anyone know why I have this problem?
AVAssetImageGenerator *generator = [[AVAssetImageGenerator alloc] initWithAsset:asset];
generator.appliesPreferredTrackTransform = YES;
generator.requestedTimeToleranceAfter = kCMTimeZero;
generator.requestedTimeToleranceBefore = kCMTimeZero;
CGSize maxSize = CGSizeMake(320, 180);
generator.maximumSize = maxSize;
AVAssetImageGeneratorCompletionHandler handler = ^(CMTime requestedTime, CGImageRef im, CMTime actualTime, AVAssetImageGeneratorResult result, NSError *error){
if (result != AVAssetImageGeneratorSucceeded) {
NSLog(#"couldn't generate thumbnail, error:%#", error);
}
UIImage *frameImage = [UIImage imageWithCGImage:im];
dispatch_async(dispatch_get_main_queue(), ^{
[_frameImageView setImage:frameImage];
});
};
[generator generateCGImagesAsynchronouslyForTimes:timeArray completionHandler:handler];
I know your issues.
It takes a lot of time for generating thumbnail because you set requestedTimeToleranceAfter and requestedTimeToleranceBefore are kCMTimeZero.
Long Answer:
If you specific TimeTolerance, it will be turned for precision rather than performance. if you just want to video thumbnail, so you don't need generate thumbnail with hight precision.
It's similar with seekToTime with tolerance. Reference from https://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/02_Playback.html#//apple_ref/doc/uid/TP40010188-CH3-SW3 , Section Seeking—Repositioning the Playhead.
Short Answer :
Just remove requestedTimeToleranceAfter and requestedTimeToleranceBefore.

Getting video snapshot for thumbnail

I am recording a video from the iPhone camera by using the AVCam code provided from apple.
After the video is recorded it is saved to the photos library.
A new view is then loaded, here I need to have an image thumbnail of the video.
I have a path to the video:
file://localhost/private/var/mobile/Applications/ED45DEFC-ABF9-4A5E-9102-21680CC1448E/tmp/output.mov
I can't seem to figure how to get the first frame of the video to use as a thumbnail.
Any help would be very appreciated and thank you for your time.
EDIT
I ended up using this, I'm not sure why it returns the image sideways?
- (UIImage*)loadImage {
AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:vidURL options:nil];
AVAssetImageGenerator *generate = [[AVAssetImageGenerator alloc] initWithAsset:asset];
NSError *err = NULL;
CMTime time = CMTimeMake(1, 60);
CGImageRef imgRef = [generate copyCGImageAtTime:time actualTime:NULL error:&err];
NSLog(#"err==%#, imageRef==%#", err, imgRef);
return [[UIImage alloc] initWithCGImage:imgRef];
}
To fix the thumbnail orientation set appliesPreferredTrackTransform to YES in the AVAssetImageGenerator instance. If you add your own video composition, you'll need to include the right transform to rotate the video as wanted.
generate.appliesPreferredTrackTransform = YES;
Remember to release the obtained image reference with CGImageRelease.
To request multiple thumbnails it's better to do asynchronously with generateCGImagesAsynchronouslyForTimes:completionHandler:.

Resources