I am trying to fetch all frames of video and converting and storing them as individual images.
I am using this code in AV Foundation Programming Guide.
the code for getting multiple images is
CMTime firstThird = CMTimeMakeWithSeconds(durationSeconds/3.0, 600);
CMTime secondThird = CMTimeMakeWithSeconds(durationSeconds*2.0/3.0, 600);
CMTime end = CMTimeMakeWithSeconds(durationSeconds, 600);
this is hard coded, but I want to convert whole video. I know I can use for loop but what to do with this durationsecond means how can I use from begging to end to get all frames?
here is my attempt
for(float f=0.0; f<=durationSeconds; f++) {
[times addObject:[NSValue valueWithCMTime:CMTimeMakeWithSeconds(durationSeconds, 600)]];
}
Any time you're about to write hundreds of lines of nearly identical code is probably a time where you need to be using a loop of some sort:
for (int currentFrame = 0; currentFrame < durationSeconds; ++currentFrame) {
CMTime currentTime = CMTimeMakeWithSeconds(i, 600);
// the rest of the code you need to create the image or whatever
}
That snippet will grab one frame per second. If you wanted to grab 30 frames per second, it'd look more like this:
const CGFloat framesPerSecond = 30.0;
for (int currentFrame = 0; currentFrame < (durationSeconds * framesPerSecond); ++currentFrame) {
CMTime currentTime = CMTimeMakeWithSeconds(currentFrame/framesPerSecond, 600);
// again, the code you need to create the image from this time
}
Just set the value of framesPerSecond to however many frames per second you want to capture.
As a disclaimer, I'm not completely familiar with this stuff, so a <= might be appropriate in the conditional statements here.
ADDENDUM: The code I've posted is only going to grab the timestamp for which to grab an image. The rest of the code should look something like this:
AVAsset *myAsset = // your asset here
AVAssetImageGenerator *imageGenerator = [[AVAssetImageGenerator alloc] initWithAsset:myAsset];
NSError *error;
CMTime actualTime;
CGImageRef currentImage = [imageGenerator copyCGImageAtTime:currentTime
actualTime:&actualTime
error:&error];
if (!error) {
[someMutableArray addObject:[[UIImage alloc] initWithCGImage:currentImage]];
}
Related
I am very upset because from last two days i am searching for editing video frame and replacing them at same time(frame actual time) with edited frame but i am unable to do that.
I have seen so many links of stackoverflow but that is not perfect for my requirement- iFrame extractor is not working.
Get image from a video frame in iPhone
In mean while, I though to extract all frames and save it in an array of dictionary with frames and respecting time and when i got a frame from running video then after editing i will run a loop for actual frame with respecting time and i will match the actual time with captured(edited frame running from video) frame and if it got then replace actual frame with edited frame and again write the video from all frames.
but to do so i used -
Get all frames of Video IOS 6
but it is crashing after extracting some images .
I have written my code like this
-(void)editMovie:(id)sender
{
float FPS=1;
NSString *videoPath = [[NSBundle mainBundle] pathForResource:#"IMG_0879" ofType:#"MOV"];
NSURL *videoURl = [NSURL fileURLWithPath:videoPath];
AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:videoURl options:nil];
AVAssetImageGenerator *generator = [[AVAssetImageGenerator alloc] initWithAsset:asset];
generator.requestedTimeToleranceAfter = kCMTimeZero;
generator.requestedTimeToleranceBefore = kCMTimeZero;
for (Float64 i = 0; i < CMTimeGetSeconds(asset.duration) * FPS ; i++){
#autoreleasepool {
CMTime time = CMTimeMake(i, FPS);
NSError *err;
CMTime actualTime;
CGImageRef image = [generator copyCGImageAtTime:time actualTime:&actualTime error:&err];
UIImage *generatedImage = [[UIImage alloc] initWithCGImage:image];
[self saveImage: generatedImage atTime:actualTime]; // Saves the image on document directory and not memory
CGImageRelease(image);
}
}
}
-(void)saveImage:(UIImage*)image atTime:(CMTime)time
{
float t=time.timescale;
NSMutableDictionary *dict=[[NSMutableDictionary alloc]init];
[dict setObject:image forKey:#"image"];
[dict setObject:[NSString stringWithFormat:#"%f",t] forKey:#"time"];
[arrFrame addObject:dict];
NSLog(#"ArrayImg=%#",arrFrame);
}
My Requrement is ->>>
I need to run a video
From running video i have to pause a video and get image(frame) at pause time.
I have to edit captured image and save or replace at actual image.
4 when i agin play the video the edited image should be in video.
Please give me any example if you all have, I have downloaded so many projects or examples from given link of stack flow or other sites but no one perfect or even fulfilling my 20 % requirement.
Please give me any example or ides if you have.
I will be obliged to you, Thanks in Advance
I am trying to go through each frame in an AVAsset and process each frame as if it were an image. I have not been able to find anything from my searches.
The task I am trying to accomplish would look like this in pseudo-code
for each frame in asset
take the frame as an image and convert to a cvMat
Process and store data of center points
Store center points in array
The only part in that pseudo-code I do not know how to write is the going though each frame and capturing it in an image.
Can anyone help?
One answer is to use AVAssetImageGenerator.
1) Load the movie file into an AVAsset object.
2) Create an AVAssetImageGenerator object.
3) Pass in an estimated time of the frame where you want to get an image back from the movie.
Setting the 2 properties requestedTimeToleranceBefore and requestedTimeToleranceAfter on the AVAssetImageGenerator object to kCMTimeZero will increase the ability to get individual frames, but increases the processing time.
However this method is slow and I have not found a faster way.
//Load the Movie from a URL
self.movieAsset = [AVAsset assetWithURL:self.movieURL];
NSArray *movieTracks = [self.movieAsset tracksWithMediaType:AVMediaTypeVideo];
AVAssetTrack *movieTrack = [movieTracks objectAtIndex:0];
//Make the image Generator
AVAssetImageGenerator *imageGenerator = [[AVAssetImageGenerator alloc] initWithAsset:self.movieAsset];
//Create a variables for the time estimation
Float64 durationSeconds = CMTimeGetSeconds(self.movieAsset.duration);
Float64 timePerFrame = 1.0 / (Float64)movieTrack.nominalFrameRate;
Float64 totalFrames = durationSeconds * movieTrack.nominalFrameRate;
//Step through the frames
for (int counter = 0; counter <= totalFrames; counter++){
CMTime actualTime;
Float64 secondsIn = ((float)counter/totalFrames)*durationSeconds;
CMTime imageTimeEstimate = CMTimeMakeWithSeconds(secondsIn, 600);
NSError *error;
CGImageRef image = [imageGenerator copyCGImageAtTime:imageTimeEstimate actualTime:&actualTime error:&error];
...Do some processing on the image
CGImageRelease(image);
}
You could simply gen each frame using AVAssetReaderTrackOutput:
let asset = AVAsset(url: inputUrl)
let reader = try! AVAssetReader(asset: asset)
let videoTrack = asset.tracks(withMediaType: .video).first!
let outputSettings = [String(kCVPixelBufferPixelFormatTypeKey): NSNumber(value: kCVPixelFormatType_32BGRA)]
let trackReaderOutput = AVAssetReaderTrackOutput(track: videoTrack,
outputSettings: outputSettings)
reader.add(trackReaderOutput)
reader.startReading()
while let sampleBuffer = trackReaderOutput.copyNextSampleBuffer() {
if let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
// do what you want
}
}
I'm working on an iOS project that capture videos and extract images from this video (for example 10 pictures every 500 ms). My problem is for short videos (few seconds). I'm extracting the same pictures 2 or 3 times. I don't have this problem with video lengths around 10 seconds and more.
When discussing with a friend, he told me it could be a problem with key frame number in video.
I'm using AVCaptureDeviceInput, AVCaptureSession and AVCaptureMovieFileOutput to take the video.
So, my questions are :
How can I figure out why i'm extracting the same pictures many times ?
Is it a key-frame number problem and is it possible to increase this value (from capture session ? from capture device ?).
EDIT :
Here is the code for the picture extract :
AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:videoUrl options:nil];
AVAssetImageGenerator *generateImg = [[AVAssetImageGenerator alloc] initWithAsset:asset];
NSMutableArray *pictList = [NSMutableArray array];
for (int i = 0; i < timeList.count; i++) {
NSError *error = NULL;
CMTime time = CMTimeMake([[timeList objectAtIndex:i] intValue], 1000);
CGImageRef refImg = [generateImg copyCGImageAtTime:time actualTime:NULL error:&error];
//NSLog(#"error==%#, Refimage==%#", error, refImg);
[pictList addObject:[[UIImage alloc] initWithCGImage:refImg]];
}
And here is the timeList print out :
(lldb) po timeList
<__NSArrayM 0x175afab0>(
0,
131,
262,
393,
524
)
Thanks in advance !
According to the documentation, there is 2 property on the AVAssetImageGenerator. These properties allow to define a tolerance for getting the nearest frame. But if I put this tolerance to zero, it generates exactly the frame I request.
Here is the corrected instanciation of the AVAssetImageGenerator :
AVAssetImageGenerator *generateImg = [[AVAssetImageGenerator alloc] initWithAsset:asset];
generateImg.requestedTimeToleranceBefore = kCMTimeZero;
generateImg.requestedTimeToleranceAfter = kCMTimeZero;
It could be slower with big videos, but in my case i don't have time stakes so it's perfect.
I guess you're running into frame rate issues if you're requesting up to 10 frames every 500ms. This could be due to low frame rate source data or it could be due to a calculation issue causing the frames requested to be too close together.
Consider getting the nominalFrameRate of the tracksWithMediaCharacteristic: of the asset and working with multiples of that frame rate.
In my project, I need to copy a chunk of each frame of a video on one unique resulting image.
Capturing video frames is not a big deal. It would be something like :
// duration is the movie lenght in s.
// frameDuration is 1/fps. (or 24fps, frameDuration = 1/24)
// player is a MPMoviePlayerController
for (NSTimeInterval i=0; i < duration; i += frameDuration) {
UIImage * image = [player thumbnailImageAtTime:i timeOption:MPMovieTimeOptionExact];
CGRect destinationRect = [self getDestinationRect:i];
[self drawImage:image inRect:destinationRect fromRect:originRect];
// UI feedback
[self performSelectorOnMainThread:#selector(setProgressValue:) withObject:[NSNumber numberWithFloat:x/totalFrames] waitUntilDone:NO];
}
The problem comes when I try to implement drawImage:inRect:fromRect: method.
I tried this code, which :
create a new CGImage with CGImageCreateWithImageInRect from the video frame to extract the chunk of image.
Make a CGContextDrawImage on the ImageContext to draw the chunk
But when the video reaches 12-14s, my iPhone 4S is announcing his third memory warning and crashes. I've profiled the app with the Leak tool, and it found no leak at all...
I'm not very strong in Quartz. Is there better optimized way to achieve this?
Finally I kept the Quartz part of my code and changed the way I retrieved the images.
Now I use AVFoundation, which is a far faster solution.
// Creating the tools : 1/ the video asset, 2/ the image generator, 3/ the composition, which helps to retrieve video properties.
AVURLAsset *asset = [[[AVURLAsset alloc] initWithURL:moviePathURL
options:[NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES], AVURLAssetPreferPreciseDurationAndTimingKey, nil]] autorelease];
AVAssetImageGenerator *generator = [[[AVAssetImageGenerator alloc] initWithAsset:asset] autorelease];
generator.appliesPreferredTrackTransform = YES; // if I omit this, the frames are rotated 90° (didn't try in landscape)
AVVideoComposition * composition = [AVVideoComposition videoCompositionWithPropertiesOfAsset:asset];
// Retrieving the video properties
NSTimeInterval duration = CMTimeGetSeconds(asset.duration);
frameDuration = CMTimeGetSeconds(composition.frameDuration);
CGSize renderSize = composition.renderSize;
CGFloat totalFrames = round(duration/frameDuration);
// Selecting each frame we want to extract : all of them.
NSMutableArray * times = [NSMutableArray arrayWithCapacity:round(duration/frameDuration)];
for (int i=0; i<totalFrames; i++) {
NSValue *time = [NSValue valueWithCMTime:CMTimeMakeWithSeconds(i*frameDuration, composition.frameDuration.timescale)];
[times addObject:time];
}
__block int i = 0;
AVAssetImageGeneratorCompletionHandler handler = ^(CMTime requestedTime, CGImageRef im, CMTime actualTime, AVAssetImageGeneratorResult result, NSError *error){
if (result == AVAssetImageGeneratorSucceeded) {
int x = round(CMTimeGetSeconds(requestedTime)/frameDuration);
CGRect destinationStrip = CGRectMake(x, 0, 1, renderSize.height);
[self drawImage:im inRect:destinationStrip fromRect:originStrip inContext:context];
}
else
NSLog(#"Ouch: %#", error.description);
i++;
[self performSelectorOnMainThread:#selector(setProgressValue:) withObject:[NSNumber numberWithFloat:i/totalFrames] waitUntilDone:NO];
if(i == totalFrames) {
[self performSelectorOnMainThread:#selector(performVideoDidFinish) withObject:nil waitUntilDone:NO];
}
};
// Launching the process...
generator.requestedTimeToleranceBefore = kCMTimeZero;
generator.requestedTimeToleranceAfter = kCMTimeZero;
generator.maximumSize = renderSize;
[generator generateCGImagesAsynchronouslyForTimes:times completionHandler:handler];
Even with very long video, it takes the time but it never crash !
In addition to Martin's answer I'd suggest shrinking the sizes of the images obtained by that call; that is, adding a property [generator.maximumSize = CGSizeMake(width,height)]; Make the images as small as possible so they wouldn't take up too much memory
How can I make the process that filtering saved video in photo library in iOS?
I got URLs of videos in the library using AssetsLibrary framework,
then, made a preview for the video.
Next step, I wanna make filtering process for video using CIFilter.
In case of real time issue, I made video filter process using AVCaptureVideoDataOutputSampleBufferDelegate.
But in case of saved video, I don't know how to make filter process.
Do I use AVAsset? If I must use that, how can I filter it? and how to save it?
always thank you.
I hope this will help you
AVAsset *theAVAsset = [[AVURLAsset alloc] initWithURL:mNormalVideoURL options:nil];
NSError *error = nil;
float width = theAVAsset.naturalSize.width;
float height = theAVAsset.naturalSize.height;
AVAssetReader *mAssetReader = [[AVAssetReader alloc] initWithAsset:theAVAsset error:&error];
[theAVAsset release];
NSArray *videoTracks = [theAVAsset tracksWithMediaType:AVMediaTypeVideo];
AVAssetTrack *videoTrack = [videoTracks objectAtIndex:0];
mPrefferdTransform = [videoTrack preferredTransform];
NSDictionary *options = [NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey];
AVAssetReaderTrackOutput* mAssetReaderOutput = [[AVAssetReaderTrackOutput alloc] initWithTrack:videoTrack outputSettings:options];
[mAssetReader addOutput:mAssetReaderOutput];
[mAssetReaderOutput release];
CMSampleBufferRef buffer = NULL;
//CMSampleBufferRef buffer = NULL;
while ( [mAssetReader status]==AVAssetReaderStatusReading ){
buffer = [mAssetReaderOutput copyNextSampleBuffer];//read next image.
}
You should have a look at CVImageBufferRef pixBuf = CMSampleBufferGetImageBuffer(sbuf) then you can have the image pointer first address, so you can add filter to pixBuf, but i find that the performance is not good, If you have any new idea,we can discuss about it further.