UIImages exported as movie error - ios

Problem
My AVAssetWriter is failing after appending 5 or so images to it using a AVAssetWriterInputPixelBufferAdaptor, and I have no idea why.
Details
This popular question helped but isn't working for my needs:
How do I export UIImage array as a movie?
Everything works as planned, I even delay the assetWriterInput until it can handle more media.
But for some reason, it always fails after 5 or so images. The images I'm using are extracted frames from a GIF
Code
Here is my iteration code:
-(void)writeImageData
{
__block int i = 0;
videoQueue = dispatch_queue_create("com.videoQueue", DISPATCH_QUEUE_SERIAL);
[self.writerInput requestMediaDataWhenReadyOnQueue:dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0) usingBlock:^{
while (self.writerInput.readyForMoreMediaData) {
if (i >= self.imageRefs.count){
[self endSession];
videoQueue = nil;
[self saveToLibraryWithCompletion:^{
NSLog(#"Saved");
}];
break;
}
if (self.writerInput.readyForMoreMediaData){
CGImageRef imageRef = (__bridge CGImageRef)self.imageRefs[i];
CVPixelBufferRef buffer = [self pixelBufferFromCGImageRef:imageRef];
CGFloat timeScale = (CGFloat)self.imageRefs.count / self.originalDuration;
BOOL accepted = [self.adaptor appendPixelBuffer:buffer withPresentationTime:CMTimeMake(i, timeScale)];
CVBufferRelease(buffer);
if (!accepted){
NSLog(#"Buffer did not add %#, index %d, timescale %f", self.writer.error, i, timeScale);
}else{
NSLog(#"Buffer did nothing wrong");
}
i++;
}
}
}];
}
My other bits of code match the code from the Link above. This is only slightly different:
-(CVPixelBufferRef)pixelBufferFromCGImageRef:(CGImageRef)image
{
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CGFloat width = 640;
CGFloat height = 640;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, width,
height, kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options,
&pxbuffer);
NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
NSParameterAssert(pxdata != NULL);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, width,
height, 8, 4*width, rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
NSParameterAssert(context);
CGContextDrawImage(context, CGRectMake(0, 0, width,
height), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return pxbuffer;
}

One thing that stands out to me is your use of CMTimeMake(adjustedTime, 1).
You need to calculate the time of each frame properly. Note that CMTime takes two integers, and passing them as floating point values in truncates them.
The second issue is that you weren't using your serial dispatch queue :)

Related

Image from CMSampleBufferRef is always white

I am trying to get each frame from the replaykit using startCaptureWithHandler.
startCaptureWithHandler returns a CMSampleBufferRef which i need to convert to an image.
Im using this method to convert to UIImage but its always white.
- (UIImage *) imageFromSampleBuffer3:(CMSampleBufferRef) sampleBuffer
{
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey, [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey, nil];
CVPixelBufferRef pxbuffer = NULL;
CVPixelBufferCreate(kCFAllocatorDefault, width, height, kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options, &pxbuffer);
CVPixelBufferLockFlags flags = (CVPixelBufferLockFlags)0;
CVPixelBufferLockBaseAddress(pxbuffer, flags);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
// CGContextRef context = CGBitmapContextCreate(pxdata, width, height, 8, CVPixelBufferGetBytesPerRow(pxbuffer), rgbColorSpace, kCGImageAlphaPremultipliedFirst);
CGContextRef context = CGBitmapContextCreate(pxdata, width, height, 8, CVPixelBufferGetBytesPerRow(pxbuffer), rgbColorSpace, kCGImageAlphaPremultipliedFirst);
CGImageRef quartzImage = CGBitmapContextCreateImage(context);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, flags);
UIImage *image = [UIImage imageWithCGImage:quartzImage scale:1.0f orientation:UIImageOrientationRight];
CGImageRelease(quartzImage);
return image;
}
Can anyone tell me where im going wrong?
sampleBuffer is 420f format and it has 2 planar.
For locking memory, CVPixelBufferLockBaseAddress(imageBuffer, 0 then 1).
For getting Y plane data, CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0).
For UV plane data, CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 1).
Do not forget unlock memory. I am not sure how to convert YUV to RGB.
On your code, you do not read image data from imageBuffer.
You only work on pxbuffer and pxdata which has no image data.

How do you scale an image using vImage in the Accelerate framework in iOS 8?

I am trying to resize a CMSampleBufferRef as quickly as possible on an iOS 8 device for use in image processing. From what I have found online, the way to do this seems to be by using the vImage API in the Accelerate framework. However, I haven't done much with the Accelerate framework and I can't quite figure out how to do this. Here is what I have so far to scale an image to 200x200:
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
CVImageBufferRef cvimgRef = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(cvimgRef,0);
void *imageData = CVPixelBufferGetBaseAddress(cvimgRef);
NSInteger width = CVPixelBufferGetWidth(cvimgRef);
NSInteger height = CVPixelBufferGetHeight(cvimgRef);
unsigned char *newData= // NOT SURE WHAT THIS SHOULD BE...
vImage_Buffer inBuff = { imageData, height, width, 4*width };
vImage_Buffer outBuff = { newData, 200, 200, 4*200 };
// NOT SURE IF THIS IS THE CORRECT METHOD... video output settings for kCVPixelBufferPixelFormatTypeKey is set to kCVPixelFormatType_32BGRA
// This seems wrong since the image scale is ARGB, not BGRA.
vImageScale_ARGB8888(inBuffer, outBuffer, NULL, kvImageNoFlags);
CVPixelBufferUnlockBaseAddress(cvimgRef,0);
}
Where outBuffer is the result. After that, I am also not sure how to convert the outBuffer back to a CVImageBufferRef for further image processing. Any suggestions would be appreciated!
vImageScale returns just a buffer data, and pay attention that buffers need to be freed.
I don't know if there is a faster way just using that out buffer but I would convert the buffer into a CGImage. Something like that taken from here so take it as a reference
vImage_CGImageFormat format = {
.bitsPerComponent = 8,
.bitsPerPixel = 32,
.colorSpace = NULL,
.bitmapInfo = (CGBitmapInfo)kCGImageAlphaFirst,
.version = 0,
.decode = NULL,
.renderingIntent = kCGRenderingIntentDefault,
};
ret = kvImageNoError;
CGImageRef destRef = vImageCreateCGImageFromBuffer(&dstBuffer, &format, NULL, NULL, kvImageNoFlags, &ret)
Later I will convert it into a CVPixelBuffer.
- (CVPixelBufferRef) pixelBufferFromCGImage: (CGImageRef) image
{
NSDictionary *options = #{
(NSString*)kCVPixelBufferCGImageCompatibilityKey : #YES,
(NSString*)kCVPixelBufferCGBitmapContextCompatibilityKey : #YES,
};
CVPixelBufferRef pxbuffer = NULL;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, CGImageGetWidth(image),
CGImageGetHeight(image), kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options,
&pxbuffer);
if (status!=kCVReturnSuccess) {
DLog(#"Operation failed");
}
NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, CGImageGetWidth(image),
CGImageGetHeight(image), 8, 4*CGImageGetWidth(image), rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
NSParameterAssert(context);
CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return pxbuffer;
}
I'm pretty sure that is possible to avoid the conversion into a CGImage and start using the buffer, but I never tried.
You have to use a resampling filter in conjunction with any vImage operations that alter image geometry: page 32, vImage Programming Guide.
- (CVPixelBufferRef)copyRenderedPixelBuffer:(CVPixelBufferRef)pixelBuffer {
CVPixelBufferLockBaseAddress( pixelBuffer, 0 );
// vImage processing
vImage_Error err;
vImage_Buffer buffer;
buffer.data = (unsigned char *)CVPixelBufferGetBaseAddress( pixelBuffer );
buffer.rowBytes = CVPixelBufferGetBytesPerRow( pixelBuffer );
buffer.width = CVPixelBufferGetWidth( pixelBuffer );
buffer.height = CVPixelBufferGetHeight( pixelBuffer );
vImageCVImageFormatRef vformat = vImageCVImageFormat_CreateWithCVPixelBuffer( pixelBuffer );
vImage_CGImageFormat cgformat = {
.bitsPerComponent = 8,
.bitsPerPixel = 32,
.bitmapInfo = kCGBitmapByteOrderDefault,
.colorSpace = NULL, //sRGB
};
const CGFloat bgColor[3] = {0.0, 0.0, 0.0};
vImageBuffer_InitWithCVPixelBuffer(&buffer, &cgformat, pixelBuffer, vformat, bgColor, kvImageNoAllocate);
vImage_Buffer outbuffer;
void *tempBuffer;
tempBuffer = malloc(CVPixelBufferGetBytesPerRow( pixelBuffer ) * CVPixelBufferGetHeight( pixelBuffer ));
outbuffer.data = tempBuffer;
outbuffer.rowBytes = CVPixelBufferGetBytesPerRow( pixelBuffer );
outbuffer.width = CVPixelBufferGetWidth( pixelBuffer );
outbuffer.height = CVPixelBufferGetHeight( pixelBuffer );
// PROCESS vIMAGE HERE
err = vImageBuffer_CopyToCVPixelBuffer(&outbuffer, &cgformat, pixelBuffer, vformat, bgColor, kvImageNoFlags);
if(err != -1)
free(tempBuffer);
CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );
return (CVPixelBufferRef)CFRetain( pixelBuffer );
}

AVAssetWriter - pixel buffer for superimposed images

I can successfully create a movie from a single still image. However I am also given an array of smaller images that I need to superimpose on top of the background image. I've tried just repeating the process of appending frames with the assetWriter, but I get errors because you can't write to the same frame you've already written to.
So, I assume you have to compose the entire pixel buffer for each frame completely before you write the frame. But how would you do that?
Here's my code that works for rendering one background image:
CGSize renderSize = CGSizeMake(320, 568);
NSUInteger fps = 30;
self.assetWriter = [[AVAssetWriter alloc] initWithURL:
[NSURL fileURLWithPath:videoOutputPath] fileType:AVFileTypeQuickTimeMovie
error:&error];
NSParameterAssert(self.assetWriter);
NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
AVVideoCodecH264, AVVideoCodecKey,
[NSNumber numberWithInt:renderSize.width], AVVideoWidthKey,
[NSNumber numberWithInt:renderSize.height], AVVideoHeightKey,
nil];
AVAssetWriterInput* videoWriterInput = [AVAssetWriterInput
assetWriterInputWithMediaType:AVMediaTypeVideo
outputSettings:videoSettings];
AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor
assetWriterInputPixelBufferAdaptorWithAssetWriterInput:videoWriterInput
sourcePixelBufferAttributes:nil];
NSParameterAssert(videoWriterInput);
NSParameterAssert([self.assetWriter canAddInput:videoWriterInput]);
videoWriterInput.expectsMediaDataInRealTime = YES;
[self.assetWriter addInput:videoWriterInput];
//Start a session:
[self.assetWriter startWriting];
[self.assetWriter startSessionAtSourceTime:kCMTimeZero];
CVPixelBufferRef buffer = NULL;
NSInteger totalFrames = 90; //3 seconds
//process the bg image
int frameCount = 0;
UIImage* resizedImage = [UIImage resizeImage:self.bgImage size:renderSize];
buffer = [self pixelBufferFromCGImage:[resizedImage CGImage]];
BOOL append_ok = YES;
int j = 0;
while (append_ok && j < totalFrames) {
if (adaptor.assetWriterInput.readyForMoreMediaData) {
CMTime frameTime = CMTimeMake(frameCount,(int32_t) fps);
append_ok = [adaptor appendPixelBuffer:buffer withPresentationTime:frameTime];
if(!append_ok){
NSError *error = self.assetWriter.error;
if(error!=nil) {
NSLog(#"Unresolved error %#,%#.", error, [error userInfo]);
}
}
}
else {
printf("adaptor not ready %d, %d\n", frameCount, j);
[NSThread sleepForTimeInterval:0.1];
}
j++;
frameCount++;
}
if (!append_ok) {
printf("error appending image %d times %d\n, with error.", frameCount, j);
}
//Finish the session:
[videoWriterInput markAsFinished];
[self.assetWriter finishWritingWithCompletionHandler:^() {
self.assetWriter = nil;
}];
- (CVPixelBufferRef)pixelBufferFromCGImage:(CGImageRef)image {
CGSize size = CGSizeMake(320,568);
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
size.width,
size.height,
kCVPixelFormatType_32ARGB,
(__bridge CFDictionaryRef) options,
&pxbuffer);
if (status != kCVReturnSuccess){
NSLog(#"Failed to create pixel buffer");
}
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, size.width,
size.height, 8, 4*size.width, rgbColorSpace,
(CGBitmapInfo)kCGImageAlphaPremultipliedFirst);
CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return pxbuffer;
}
Again, the question is how to create a pixel buffer for a background image and an array of N small images that will be layered on top of the bg image. The next step after this will be to also superimposed a small video.
You can add the pixel info from the image list over the pixel buffer.
This example code shows how to add BGRA data over a ARGB pixelbuffer.
// Try to create a pixel buffer with the image mat
uint8_t* videobuffer = m_imageBGRA.data;
// From image buffer (BGRA) to pixel buffer
CVPixelBufferRef pixelBuffer = NULL;
CVReturn status = CVPixelBufferCreate (NULL, m_width, m_height, kCVPixelFormatType_32ARGB, NULL, &pixelBuffer);
if ((pixelBuffer == NULL) || (status != kCVReturnSuccess))
{
NSLog(#"Error CVPixelBufferPoolCreatePixelBuffer[pixelBuffer=%#][status=%d]", pixelBuffer, status);
return;
}
else
{
uint8_t *videobuffertmp = videobuffer;
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
GLubyte *pixelBufferData = (GLubyte *)CVPixelBufferGetBaseAddress(pixelBuffer);
// Add data for all the pixels in the image
for( int row=0 ; row<m_width ; ++row )
{
for( int col=0 ; col<m_height ; ++col )
{
memcpy(&pixelBufferData[0], &videobuffertmp[3], sizeof(uint8_t)); // alpha
memcpy(&pixelBufferData[1], &videobuffertmp[2], sizeof(uint8_t)); // red
memcpy(&pixelBufferData[2], &videobuffertmp[1], sizeof(uint8_t)); // green
memcpy(&pixelBufferData[3], &videobuffertmp[0], sizeof(uint8_t)); // blue
// Move the buffer pointer to the next pixel
pixelBufferData += 4*sizeof(uint8_t);
videobuffertmp += 4*sizeof(uint8_t);
}
}
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
}
So, in this example, the data into a image (videobuffer) is added to the pixel buffer. Usually, the pixel data is stored in a single row, so for each pixel, we have 4 bytes (represented as 'uint8_t' in this case): First for blue, then green, next red and the last for the alpha value (remember that the original image is in BGRA format).
The pixel buffer works in the same way, so the data is stored in a sigle row (ARGB in this case, as defined with 'kCVPixelFormatType_32ARGB' parameter).
This piece of code reorders the pixel data to match with the pixelbuffer configuration:
memcpy(&pixelBufferData[0], &videobuffertmp[3], sizeof(uint8_t)); // alpha
memcpy(&pixelBufferData[1], &videobuffertmp[2], sizeof(uint8_t)); // red
memcpy(&pixelBufferData[2], &videobuffertmp[1], sizeof(uint8_t)); // green
memcpy(&pixelBufferData[3], &videobuffertmp[0], sizeof(uint8_t)); // blue
And once we have the pixel added, we can move forward a pixel by:
// Move the buffer pointer to the next pixel
pixelBufferData += 4*sizeof(uint8_t);
videobuffertmp += 4*sizeof(uint8_t);
Moving the pointers 4 bytes forward.
If your images are smaller, you can add them in a smaller region, or define an 'if' using the alpha value as target data. For example:
// Add data for all the pixels in the image
for( int row=0 ; row<m_width ; ++row )
{
for( int col=0 ; col<m_height ; ++col )
{
if( videobuffertmp[3] > 10 ) // check alpha channel
{
memcpy(&pixelBufferData[0], &videobuffertmp[3], sizeof(uint8_t)); // alpha
memcpy(&pixelBufferData[1], &videobuffertmp[2], sizeof(uint8_t)); // red
memcpy(&pixelBufferData[2], &videobuffertmp[1], sizeof(uint8_t)); // green
memcpy(&pixelBufferData[3], &videobuffertmp[0], sizeof(uint8_t)); // blue
}
// Move the buffer pointer to the next pixel
pixelBufferData += 4*sizeof(uint8_t);
videobuffertmp += 4*sizeof(uint8_t);
}
}

Why AVAssetWriter inflates video file?

Strange problem. I take frames from a video file (.mov) and write them with AVAssetWriter to another file without any explicit processing. Actually I just copy the frame from one memory buffer to another and them flush them through PixelbufferAdaptor. Then I take the resulting file, delete the original file, put the resulting file instead the original and do the same operation. Interesting thing is that the size of the file constantly grows! Can somebody explain why?
if(adaptor.assetWriterInput.readyForMoreMediaData==YES) {
CVImageBufferRef cvimgRef=nil;
CMTime lastTime=CMTimeMake(fcounter++, 30);
CMTime presentTime=CMTimeAdd(lastTime, frameTime);
CMSampleBufferRef framebuffer=nil;
CGImageRef frameImg=nil;
if ( [asr status]==AVAssetReaderStatusReading ){
framebuffer = [asset_reader_output copyNextSampleBuffer];
frameImg = [self imageFromSampleBuffer:framebuffer withColorSpace:rgbColorSpace];
}
if(frameImg && screenshot){
//CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(framebuffer);
CVReturn stat= CVPixelBufferLockBaseAddress(screenshot, 0);
pxdata=CVPixelBufferGetBaseAddress(screenshot);
bufferSize = CVPixelBufferGetDataSize(screenshot);
// Get the number of bytes per row for the pixel buffer.
bytesPerRow = CVPixelBufferGetBytesPerRow(screenshot);
// Get the pixel buffer width and height.
width = CVPixelBufferGetWidth(screenshot);
height = CVPixelBufferGetHeight(screenshot);
// Create a Quartz direct-access data provider that uses data we supply.
CGDataProviderRef dataProvider = CGDataProviderCreateWithData(NULL, pxdata, bufferSize, NULL);
CGImageAlphaInfo ai=CGImageGetAlphaInfo(frameImg);
size_t bpx=CGImageGetBitsPerPixel(frameImg);
CGColorSpaceRef fclr=CGImageGetColorSpace(frameImg);
// Create a bitmap image from data supplied by the data provider.
CGImageRef cgImage = CGImageCreate(width, height, 8, 32, bytesPerRow,rgbColorSpace, kCGImageAlphaNoneSkipLast | kCGBitmapByteOrder32Big,dataProvider, NULL, true, kCGRenderingIntentDefault);
CGDataProviderRelease(dataProvider);
stat= CVPixelBufferLockBaseAddress(finalPixelBuffer, 0);
pxdata=CVPixelBufferGetBaseAddress(finalPixelBuffer);
bytesPerRow = CVPixelBufferGetBytesPerRow(finalPixelBuffer);
CGContextRef context = CGBitmapContextCreate(pxdata, imgsize.width,imgsize.height, 8, bytesPerRow, rgbColorSpace, kCGImageAlphaNoneSkipLast);
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(frameImg), CGImageGetHeight(frameImg)), frameImg);
//CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
//CGImageRef myMaskedImage;
const float myMaskingColors[6] = { 0, 0, 0, 1, 0, 0 };
CGImageRef myColorMaskedImage = CGImageCreateWithMaskingColors (cgImage, myMaskingColors);
//CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(myColorMaskedImage), CGImageGetHeight(myColorMaskedImage)), myColorMaskedImage);
[adaptor appendPixelBuffer:finalPixelBuffer withPresentationTime:presentTime];}
well, the mystery seems to be solved. The problem was in inappropriate codec configuration.
This is set of configuration options I use now and it seems to do the work:
NSDictionary *codecSettings = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:1100000], AVVideoAverageBitRateKey,
[NSNumber numberWithInt:5],AVVideoMaxKeyFrameIntervalKey,
nil];
NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
AVVideoCodecH264, AVVideoCodecKey,
[NSNumber numberWithInt:[SharedApplicationData sharedData].overlayView.frame.size.width], AVVideoWidthKey,
[NSNumber numberWithInt:[SharedApplicationData sharedData].overlayView.frame.size.height], AVVideoHeightKey,
codecSettings,AVVideoCompressionPropertiesKey,
nil];
AVAssetWriterInput* writerInput = [AVAssetWriterInput
assetWriterInputWithMediaType:AVMediaTypeVideo
outputSettings:videoSettings];
Now the file size still grows but at much slower pace. There is a tradeoff between the file size and the video quality - size reduction affects the quality.

IOS 5: UIView converted to video and the resulting video is corrupted

I want to grab the UIView, convert to image and than store that image in a video file (.mp4).
I use the next portion of code which grabs the image and puts it to pixel buffer:
BOOL appended;
if(input.readyForMoreMediaData==YES){
//grab the view and convert it into image
CGSize imgsize=self.imageSource.frame.size;
UIGraphicsBeginImageContext(imgsize);
[self.imageSource.layer renderInContext:UIGraphicsGetCurrentContext()];
UIImage* grabbedImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
CVReturn cvErr = kCVReturnSuccess;
CGImageRef image = (CGImageRef) [grabbedImage CGImage];
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, imgsize.width,
imgsize.height, kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options,
&pxbuffer);
NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
NSParameterAssert(pxdata != NULL);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, imgsize.width,
imgsize.height, 8, 4*imgsize.width, rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
NSParameterAssert(context);
CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
appended = [pxlBufAdaptor appendPixelBuffer:pxbuffer withPresentationTime:presentationTime];
CVBufferRelease(pxbuffer );
}
The problem is that the resulting video contains the corrupted image - all pixels are offset . It looks like the memory is filled with bytes with some offset and that offset corrupts the presentation.
How can this be fixed?
I would like to have any glue or direction.
Thanks in advance.
This looks suspicious:
CGContextRef context = CGBitmapContextCreate(pxdata, imgsize.width,
imgsize.height, 8, 4*imgsize.width, rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
You are computing the bytesPerRow parameter based on the image width, instead of asking pxbuffer for its bytes-per-row. Try this:
CGContextRef context = CGBitmapContextCreate(pxdata, imgsize.width,
imgsize.height, 8, CVPixelBufferGetBytesPerRow(pxbuffer),
rgbColorSpace, kCGImageAlphaNoneSkipFirst);
Also, it seems inefficient to create a bitmap graphics context with UIGraphicsGetCurrentContext, render the layer into the context, get an image from the context, destroy the context, create a pixel buffer, create a bitmap graphics context using the pixel buffer, and draw the image of the layer into the new context. Why not replace your CGContextDrawImage call with [self.imageSource.layer renderInContext:context]?

Resources