How do i export images as video in Swift? - ios

i am making an app in Swift (i have the last xcode update) that has to generate a video from some images.
I got the code from this answer How do I export UIImage array as a movie?
I call the function like this:
let size = CGSize(width: 1280, height: 720)
let pathVideo = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true)
let percorsoVideo = pathVideo[0]
writeImagesAsMovie(arrayImmagini, videoPath: percorsoVideo+"/prova.mp4", videoSize: size, videoFPS: 1)
"arrayImmagini" is defined literally like this:
var arrayImmagini = [UIImage(imageLiteral: "Frames/turtle/turtle0.jpg"), UIImage(imageLiteral: "Frames/turtle/turtle1.jpg"), ...]
When i try to run the code i get a completely black video and xcode gives me this 2 errors as many times as many images there are in the array:
Sep 5 09:24:15 Prova[1554] <Error>: CGBitmapContextCreate: invalid data bytes/row: should be at least 7680 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.
Sep 5 09:24:15 Prova[1554] <Error>: CGContextDrawImage: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
Reading the documentation about CGBitmapContextCreate, i tried to call it differently:
func fillPixelBufferFromImage(image: UIImage, pixelBuffer: CVPixelBufferRef) {
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
// Create CGBitmapContext
let context = CGBitmapContextCreate(
nil,
Int(image.size.width),
Int(image.size.height),
8,
0,
rgbColorSpace,
CGImageAlphaInfo.PremultipliedFirst.rawValue
)
// Draw image into context
CGContextDrawImage(context, CGRectMake(0, 0, image.size.width, image.size.height), image.CGImage)
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
}
Instead of:
func fillPixelBufferFromImage(image: UIImage, pixelBuffer: CVPixelBufferRef) {
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
// Create CGBitmapContext
let context = CGBitmapContextCreate(
pixelData,
Int(image.size.width),
Int(image.size.height),
8,
CVPixelBufferGetBytesPerRow(pixelBuffer),
rgbColorSpace,
CGImageAlphaInfo.PremultipliedFirst.rawValue
)
// Draw image into context
CGContextDrawImage(context, CGRectMake(0, 0, image.size.width, image.size.height), image.CGImage)
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
}
This made xcode stop giving me errors but i still get a black video.
Please help me, i'm new to app development and even newer to AVFoundation, i don't have any clues about how to solve it by myself.
Thank you!

After many attempts to make this work i found out what the problem was.
You can't give a video size more little than the pictures' size.
Once i did this, everything worked:
let size = CGSize(width: 1920, height: 1280)

Related

CGContext.init for ARGB image returns nil

I'm trying to create a CGContext and fill it with an array of pixels with an ARGB format. I've successfully created the pixel array, but when I try to create the CGContext with CGColorSpaceCreateDeviceRGB and CGImageAlphaInfo.first, it returns nil.
func generateBitmapImage8bit() -> CGImage {
let width = params[0]
let height = params[1]
let bitmapBytesPerRow = width * 4
let context = CGContext(data: nil,
width: width,
height: height,
bitsPerComponent: 8,
bytesPerRow: bitmapBytesPerRow,
space: CGColorSpaceCreateDeviceRGB(), //<-
bitmapInfo: CGImageAlphaInfo.first.rawValue)
context!.data!.storeBytes(of: rasterArray, as: [Int].self)
let image = context!.makeImage()
return image!
}
Please refer to the Supported Pixel Formats.
Seems like you use incorrect configuration (CGImageAlphaInfo.first). For 8 bitsPerComponent and RGB color space you have only the following valid alpha options:
kCGImageAlphaNoneSkipFirst
kCGImageAlphaNoneSkipLast
kCGImageAlphaPremultipliedFirst
kCGImageAlphaPremultipliedLast
You can also try setting CGBITMAP_CONTEXT_LOG_ERRORS environment variable in your scheme to get more information in runtime.

How to Create UIImage with kCVPixelFormatType_32BGRA formatted Buffer using Swift

We are creating UIImage from a buffer using following code
let context = CGContext(data: baseAddress, width: frameWidth, height: frameHeight, bitsPerComponent: 8, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue)!
let cgImage0 = context.makeImage()
let uiImage0 = UIImage(cgImage: cgImage0!)
The problem is we can create color image buffer using AVCaptureVideoDataOutput() only with kCVPixelFormatType_32BGRA format which is the most similar format to kCVPixelFormatType_32RGBA
At some point we need UIImage and we needed to convert buffer to UIImage and because UIImage is RGBA formatted colors has become distorted.
How can we get UIImage from buffer in kCVPixelFormatType_32RGBA format.
By making
var bitmapInfo = CGBitmapInfo.byteOrder32Little.rawValue
bitmapInfo |= CGImageAlphaInfo.premultipliedFirst.rawValue & CGBitmapInfo.alphaInfoMask.rawValue
instead of
var bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)
But getting this from Apple document is very hard.

UIImages to Video with AVAssetWriter produces black frames flickering for < 800 images

I'm stuck for weeks!
I tried all of the solutions out there on stackoverflow which they work fine for around ~ 0 to 500 images or 50 seconds on 12 fps, but for longer videos there's black flickering after ~50 seconds depending on what iphone,
with iphone 6 happens after 50 sec on iphone 8 it happens after 1:30
I tried the accepted answer from this link writeImagesAsMovie() function
sometimes I get memory allocation problem, sometimes I don't,
But the black flickering persists.
// MARK: - Fill Pixel Buffer -
private func fillPixelBufferFromImage(image: UIImage, pixelBuffer: CVPixelBuffer) {
CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
// Create CGBitmapContext
let context = CGContext(
data: pixelData,
width: Int(720),
height: Int(720),
bitsPerComponent: 8,
bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer),
space: rgbColorSpace,
bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue
)
// Draw image into context"
context?.draw(image.cgImage!, in: CGRect(x: 0, y: 0, width: 720, height: 720))
CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
//UIGraphicsEndImageContext()
}
Things that helped getting the desired result...
using UIImageJPEGRepresentation instead of PNG, the dimensions have to be multiple of 16...
:) and newer iphones can handle more images

invalid data bytes/row:CGBitmapContextCreate: CGContextDrawImage: invalid context 0x0

I'm trying to convert array of images into a video file. In the process I have to fill pixel buffer from selected images. Here is the code snippet:
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer)
let bitmapInfo:CGBitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.PremultipliedFirst.rawValue)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
print("\npixel buffer width: \(CVPixelBufferGetWidth(pixelBuffer))\n")
print("\nbytes per row: \(CVPixelBufferGetBytesPerRow(pixelBuffer))\n")
let context = CGBitmapContextCreate(
pixelData,
Int(image.size.width),
Int(image.size.height),
CGImageGetBitsPerComponent(image.CGImage),
CVPixelBufferGetBytesPerRow(pixelBuffer),
rgbColorSpace,
bitmapInfo.rawValue
)
CGContextDrawImage(context, CGRectMake(0, 0, image.size.width, image.size.height), image.CGImage)
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
After executing these lines I get the following message in xcode:
CGBitmapContextCreate: invalid data bytes/row: should be at least 13056 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.CGContextDrawImage: invalid context 0x0.
After debugging I get the following value:
CVPixelBufferGetWidth(pixelBuffer) // value 480
CVPixelBufferGetBytesPerRow(pixelBuffer) // value 1920
What should I do to get valid data bytes/row? What are the 3 components specified in console log? I saw similar questions in stackoverflow but nothing helped in my case.

CGContextDrawImage error, how to allocate the buffer?

I am trying to store image data in buffer in my app so i will be able to use it however I get EXC_BAD_ACCESS error on CGContextDrawImage line.
Here is the code i am using:
resizedImage.Array() // resizedImage - resized image of 280x140 pixels size
func Array() {
let dataBuffer = UnsafeMutablePointer<CUnsignedChar>.alloc(156800) //alloc(156800)
memset(dataBuffer, 0, 156800)
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGBitmapInfo(rawValue: CGBitmapInfo.ByteOrder32Big.rawValue | CGImageAlphaInfo.PremultipliedLast.rawValue)
let context = CGBitmapContextCreate(dataBuffer, 280, 140, 8, 156800, colorSpace, bitmapInfo.rawValue)
let imageRef = CGImageCreateWithImageInRect(CGImage, CGRectMake(0, 0, 280, 140))
let rectMake = CGRectMake(0, 0, 280, 140)
CGContextDrawImage(context, rectMake, imageRef)
return
}
When trying to set buffer and bytes as null and 0 as in apple documentation for automatic memory allocation app doesn't crash, but it gives this errors:
Sep 17 17:18:33 VideoTester[4846] <Error>: CGBitmapContextCreate: invalid data bytes/row: should be at least 1120 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedLast.
Sep 17 17:18:33 VideoTester[4846] <Error>: CGContextDrawImage: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
Seems like I am missing something but can't really figure it out, any advice is needed, thank you!
Got this working with a help of peacer212, everything seems to work as needed without any errors, here is a working code:
var dataBuffer = [UInt8](count: 280*140 * 4, repeatedValue: 0)
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGBitmapInfo(rawValue: CGBitmapInfo.ByteOrder32Big.rawValue | CGImageAlphaInfo.PremultipliedLast.rawValue)
let context = CGBitmapContextCreate(&dataBuffer, 280, 140, 8, 1120, colorSpace, bitmapInfo.rawValue)
let imageRef = CGImageCreateWithImageInRect(CGImage, CGRectMake(0, 0, 280, 140))
let rectMake = CGRectMake(0, 0, 280, 140)
CGContextDrawImage(context, rectMake, imageRef)

Resources