Frame Duration time - UIImage array to movie - ios

I have successfully completed the part "Export UIImage array as movie" through this code. In the exported video every image is shown for 1 second. But I need, every image will be shown for 5 seconds in the exported video.
What is the minimal change I need to do so ?
Here is my code...
var outputSize = CGSize(width: 1920, height: 1280)
func build(outputSize outputSize: CGSize) {
let fileManager = NSFileManager.defaultManager()
let urls = fileManager.URLsForDirectory(.DocumentDirectory, inDomains: .UserDomainMask)
guard let documentDirectory: NSURL = urls.first else {
fatalError("documentDir Error")
}
let videoOutputURL = documentDirectory.URLByAppendingPathComponent("OutputVideo.mp4")
if NSFileManager.defaultManager().fileExistsAtPath(videoOutputURL!.path!) {
do {
try NSFileManager.defaultManager().removeItemAtPath(videoOutputURL!.path!)
} catch {
fatalError("Unable to delete file: \(error) : \(#function).")
}
}
guard let videoWriter = try? AVAssetWriter(URL: videoOutputURL!, fileType: AVFileTypeMPEG4) else {
fatalError("AVAssetWriter error")
}
let outputSettings = [AVVideoCodecKey : AVVideoCodecH264, AVVideoWidthKey : NSNumber(float: Float(outputSize.width)), AVVideoHeightKey : NSNumber(float: Float(outputSize.height))]
guard videoWriter.canApplyOutputSettings(outputSettings, forMediaType: AVMediaTypeVideo) else {
fatalError("Negative : Can't apply the Output settings...")
}
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: outputSettings)
let sourcePixelBufferAttributesDictionary = [kCVPixelBufferPixelFormatTypeKey as String : NSNumber(unsignedInt: kCVPixelFormatType_32ARGB), kCVPixelBufferWidthKey as String: NSNumber(float: Float(outputSize.width)), kCVPixelBufferHeightKey as String: NSNumber(float: Float(outputSize.height))]
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: videoWriterInput, sourcePixelBufferAttributes: sourcePixelBufferAttributesDictionary)
if videoWriter.canAddInput(videoWriterInput) {
videoWriter.addInput(videoWriterInput)
}
if videoWriter.startWriting() {
videoWriter.startSessionAtSourceTime(kCMTimeZero)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = dispatch_queue_create("mediaInputQueue", nil)
videoWriterInput.requestMediaDataWhenReadyOnQueue(media_queue, usingBlock: { () -> Void in
let fps: Int32 = 1
let frameDuration = CMTimeMake(1, fps)
var frameCount: Int64 = 0
var appendSucceeded = true
while (!self.choosenPhotos.isEmpty) {
if (videoWriterInput.readyForMoreMediaData) {
let nextPhoto = self.choosenPhotos.removeAtIndex(0)
let lastFrameTime = CMTimeMake(frameCount, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
var pixelBuffer: CVPixelBuffer? = nil
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pixelBufferAdaptor.pixelBufferPool!, &pixelBuffer)
if let pixelBuffer = pixelBuffer where status == 0 {
let managedPixelBuffer = pixelBuffer
CVPixelBufferLockBaseAddress(managedPixelBuffer, CVPixelBufferLockFlags(rawValue: CVOptionFlags(0)))
let data = CVPixelBufferGetBaseAddress(managedPixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGBitmapContextCreate(data, Int(self.outputSize.width), Int(self.outputSize.height), 8, CVPixelBufferGetBytesPerRow(managedPixelBuffer), rgbColorSpace, CGImageAlphaInfo.PremultipliedFirst.rawValue)
CGContextClearRect(context!, CGRectMake(0, 0, CGFloat(self.outputSize.width), CGFloat(self.outputSize.height)))
let horizontalRatio = CGFloat(self.outputSize.width) / nextPhoto.size.width
let verticalRatio = CGFloat(self.outputSize.height) / nextPhoto.size.height
let aspectRatio = min(horizontalRatio, verticalRatio) // ScaleAspectFit
let newSize:CGSize = CGSizeMake(nextPhoto.size.width * aspectRatio, nextPhoto.size.height * aspectRatio)
let x = newSize.width < self.outputSize.width ? (self.outputSize.width - newSize.width) / 2 : 0
let y = newSize.height < self.outputSize.height ? (self.outputSize.height - newSize.height) / 2 : 0
CGContextDrawImage(context!, CGRectMake(x, y, newSize.width, newSize.height), nextPhoto.CGImage!)
CVPixelBufferUnlockBaseAddress(managedPixelBuffer, CVPixelBufferLockFlags(rawValue: CVOptionFlags(0)))
appendSucceeded = pixelBufferAdaptor.appendPixelBuffer(pixelBuffer, withPresentationTime: presentationTime)
} else {
print("Failed to allocate pixel buffer")
appendSucceeded = false
}
}
if !appendSucceeded {
break
}
frameCount += 1
}
videoWriterInput.markAsFinished()
videoWriter.finishWritingWithCompletionHandler { () -> Void in
print("FINISHED!!!!!")
}
})
}
}

Just change:
let frameDuration = CMTimeMake(5, fps)
And change this line from:
let lastFrameTime = CMTimeMake(frameCount, fps)
To
let lastFrameTime = CMTimeMake(frameCount*5, fps)
Or
let lastFrameTime = frameDuration
Hope this helps.

let lastFrameTime = CMTimeMake(frameCount * 5, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
CMTime t2 = CMTimeMake(2, 1); // 2 seconds It represent value/timeScale.

Related

Swift | Creating video out of image array

I build a App, where user can make a Video of a few images.
I took the code from this post: Making video from UIImage array with different transition animations
Everything is working fine, but I have one big problem and I don't know how to solve it.
When the video was created, the image is not filling the hole screen and sometimes it turns to another side or the image is cut in half.
First image is the original image and the second image is the created video with the problems I have.
I really need some help.
var outputSize = CGSize(width: 1920 , height: 1280)
var imagesPerSecond: TimeInterval = 0
var fps: Int32 = 0
var selectedPhotosArray = [UIImage]()
var imageArrayToVideoURL = NSURL()
var asset: AVAsset!
var imageCount = 1
My Code
func buildVideoFromImageArray() {
for image in arrayOfImages {
selectedPhotosArray.append(image)
}
imageArrayToVideoURL = NSURL(fileURLWithPath: NSHomeDirectory() + "/Documents/video1.MP4")
removeFileAtURLIfExists(url: imageArrayToVideoURL)
guard let videoWriter = try? AVAssetWriter(outputURL: imageArrayToVideoURL as URL, fileType: AVFileType.mp4) else {
fatalError("AVAssetWriter error")
}
let outputSettings = [AVVideoCodecKey : AVVideoCodecType.h264, AVVideoWidthKey : NSNumber(value: Float(outputSize.width)), AVVideoHeightKey : NSNumber(value: Float(outputSize.height))] as [String : Any]
guard videoWriter.canApply(outputSettings: outputSettings, forMediaType: AVMediaType.video) else {
fatalError("Negative : Can't applay the Output settings...")
}
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: outputSettings)
let sourcePixelBufferAttributesDictionary = [kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32ARGB), kCVPixelBufferWidthKey as String: NSNumber(value: Float(outputSize.width)), kCVPixelBufferHeightKey as String: NSNumber(value: Float(outputSize.height))]
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: videoWriterInput, sourcePixelBufferAttributes: sourcePixelBufferAttributesDictionary)
if videoWriter.canAdd(videoWriterInput) {
videoWriter.add(videoWriterInput)
}
if videoWriter.startWriting() {
let zeroTime = CMTimeMake(value: Int64(imagesPerSecond),timescale: self.fps)
videoWriter.startSession(atSourceTime: zeroTime)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = DispatchQueue(label: "mediaInputQueue")
videoWriterInput.requestMediaDataWhenReady(on: media_queue, using: { () -> Void in
//let fps: Int32 = 1
let framePerSecond: Int64 = Int64(self.imagesPerSecond)
let frameDuration = CMTimeMake(value: Int64(self.imagesPerSecond), timescale: self.fps)
var frameCount: Int64 = 0
var appendSucceeded = true
while (!self.selectedPhotosArray.isEmpty) { // wird so lange ausgeführt, bis noch etwas im Array steht
if (videoWriterInput.isReadyForMoreMediaData) {
let nextPhoto = self.selectedPhotosArray.remove(at: 0) // foto wird aus dem selectedPhotosArray gelöscht
let lastFrameTime = CMTimeMake(value: frameCount * framePerSecond, timescale: self.fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
var pixelBuffer: CVPixelBuffer? = nil
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pixelBufferAdaptor.pixelBufferPool!, &pixelBuffer)
if let pixelBuffer = pixelBuffer, status == 0 {
let managedPixelBuffer = pixelBuffer
CVPixelBufferLockBaseAddress(managedPixelBuffer, CVPixelBufferLockFlags(rawValue: CVOptionFlags(0)))
let data = CVPixelBufferGetBaseAddress(managedPixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGContext(data: data, width: Int(self.outputSize.width), height: Int(self.outputSize.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(managedPixelBuffer), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue)
context!.clear(CGRect(x: 0, y: 0, width: CGFloat(self.outputSize.width), height: CGFloat(self.outputSize.height)))
let horizontalRatio = CGFloat(self.outputSize.width) / nextPhoto.size.width
let verticalRatio = CGFloat(self.outputSize.height) / nextPhoto.size.height
//let aspectRatio = max(horizontalRatio, verticalRatio) // ScaleAspectFill
let aspectRatio = min(horizontalRatio, verticalRatio) // ScaleAspectFit
let newSize: CGSize = CGSize(width: nextPhoto.size.width * aspectRatio, height: nextPhoto.size.height * aspectRatio)
let x = newSize.width < self.outputSize.width ? (self.outputSize.width - newSize.width) / 2 : 0
let y = newSize.height < self.outputSize.height ? (self.outputSize.height - newSize.height) / 2 : 0
context?.draw(nextPhoto.cgImage!, in: CGRect(x: x, y: y, width: newSize.width, height: newSize.height))
CVPixelBufferUnlockBaseAddress(managedPixelBuffer, CVPixelBufferLockFlags(rawValue: CVOptionFlags(0)))
appendSucceeded = pixelBufferAdaptor.append(pixelBuffer, withPresentationTime: presentationTime)
} else {
print("Failed to allocate pixel buffer")
appendSucceeded = false
}
}
if !appendSucceeded {
break
}
frameCount += 1
}
videoWriterInput.markAsFinished()
videoWriter.finishWriting { () -> Void in
print("-----video1 url = \(self.imageArrayToVideoURL)")
//self.asset = AVAsset(url: self.imageArrayToVideoURL as URL)
PHPhotoLibrary.shared().performChanges({
PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: self.imageArrayToVideoURL as URL)
}) { saved, error in
if saved {
let fetchOptions = PHFetchOptions()
fetchOptions.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: false)]
let fetchResult = PHAsset.fetchAssets(with: .video, options: fetchOptions).firstObject
// fetchResult is your latest video PHAsset
// To fetch latest image replace .video with .image
}
}
}
})
}
}
func removeFileAtURLIfExists(url: NSURL) {
if let filePath = url.path {
let fileManager = FileManager.default
if fileManager.fileExists(atPath: filePath) {
do{
try fileManager.removeItem(atPath: filePath)
} catch let error as NSError {
print("Couldn't remove existing destination file: \(error)")
}
}
}
}`

Video made from image collection takes a lot of time to play

I am trying to make a video editor in which I have done the following:
1)Collected photos from user gallery.
2)Converted that array of photos in videos .
3)Added animations to videos.
4)Played Videos
The code i have done for that are as follows:-
1)Collected photos from user gallery.
func openImagePicker(){
let customColor = UIColor.init(red: 64.0/255.0, green: 0.0, blue: 144.0/255.0, alpha: 1.0)
let customCameraColor = UIColor.init(red: 86.0/255.0, green: 1.0/255.0, blue: 236.0/255.0, alpha: 1.0)
pickerViewController.numberOfPhotoToSelect = 5; pickerViewController.theme.titleLabelTextColor = UIColor.white
pickerViewController.theme.navigationBarBackgroundColor = customColor
pickerViewController.theme.tintColor = UIColor.white
pickerViewController.theme.orderTintColor = customCameraColor
pickerViewController.theme.cameraVeilColor = customCameraColor
pickerViewController.theme.cameraIconColor = UIColor.white
pickerViewController.theme.statusBarStyle = .lightContent
self.yms_presentCustomAlbumPhotoView(pickerViewController, delegate: self)
}
func photoPickerViewController(_ picker: YMSPhotoPickerViewController!, didFinishPickingImages photoAssets: [PHAsset]!) {
picker.dismiss(animated: true) {
self.selectedImageArray = NSMutableArray()
let imageManager = PHImageManager.init()
let options = PHImageRequestOptions.init()
options.deliveryMode = .highQualityFormat
options.resizeMode = .exact
options.isSynchronous = true
for asset: PHAsset in photoAssets
{
let targetSize = CGSize(width:self.view.frame.size.width
, height:self.view.frame.size.width)
imageManager.requestImage(for: asset, targetSize:targetSize, contentMode: .aspectFill, options: options, resultHandler: { (image, info) in
self.selectedImageArray.add(image!)
})
}
let imageVideaMakerController = self.storyboard?.instantiateViewController(withIdentifier: "VideoEditorController") as! VideoEditorController
imageVideaMakerController.selectedImageArray = self.selectedImageArray as! [UIImage]
self.navigationController!.pushViewController(imageVideaMakerController, animated: true)
}
}
2)Converted that array of photos in videos .
override func viewDidAppear(_ animated: Bool) {
self.navigationController?.navigationBar.isHidden = false
setUpInitialView()
collectionView.reloadData()
}
//MARK:- Custom Methods
func setUpInitialView(){
let loadingNotification = MBProgressHUD.showAdded(to: view, animated: true)
loadingNotification.mode = MBProgressHUDMode.indeterminate
loadingNotification.label.text = "Loading"
buildVideoFromImageArray()
//filterScrollContents()
}
func buildVideoFromImageArray() {
imageArrayToVideoURL = NSURL(fileURLWithPath: NSHomeDirectory() + "/Documents/video1.MP4")
removeFileAtURLIfExists(url: imageArrayToVideoURL)
guard let videoWriter = try? AVAssetWriter(outputURL: imageArrayToVideoURL as URL, fileType: AVFileType.mp4) else {
fatalError("AVAssetWriter error")
}
let outputSettings = [AVVideoCodecKey : AVVideoCodecH264, AVVideoWidthKey : NSNumber(value: Float(outputSize.width)), AVVideoHeightKey : NSNumber(value: Float(outputSize.height))] as [String : Any]
guard videoWriter.canApply(outputSettings: outputSettings, forMediaType: AVMediaType.video) else {
fatalError("Negative : Can't apply the Output settings...")
}
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: outputSettings)
let sourcePixelBufferAttributesDictionary = [kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32ARGB), kCVPixelBufferWidthKey as String: NSNumber(value: Float(outputSize.width)), kCVPixelBufferHeightKey as String: NSNumber(value: Float(outputSize.height))]
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: videoWriterInput, sourcePixelBufferAttributes: sourcePixelBufferAttributesDictionary)
if videoWriter.canAdd(videoWriterInput) {
videoWriter.add(videoWriterInput)
}
if videoWriter.startWriting() {
let zeroTime = CMTimeMake(Int64(imagesPerSecond),Int32(1))
videoWriter.startSession(atSourceTime: zeroTime)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = DispatchQueue(label: "mediaInputQueue")
videoWriterInput.requestMediaDataWhenReady(on: media_queue, using: { () -> Void in
let fps: Int32 = 1
let framePerSecond: Int64 = Int64(self.imagesPerSecond)
let frameDuration = CMTimeMake(Int64(self.imagesPerSecond), fps)
var frameCount: Int64 = 0
var appendSucceeded = true
var newImageArr = self.selectedImageArray
while (!newImageArr.isEmpty) {
if (videoWriterInput.isReadyForMoreMediaData) {
let nextPhoto = newImageArr.remove(at: 0)
let lastFrameTime = CMTimeMake(frameCount * framePerSecond, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
var pixelBuffer: CVPixelBuffer? = nil
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pixelBufferAdaptor.pixelBufferPool!, &pixelBuffer)
if let pixelBuffer = pixelBuffer, status == 0 {
let managedPixelBuffer = pixelBuffer
CVPixelBufferLockBaseAddress(managedPixelBuffer, CVPixelBufferLockFlags(rawValue: CVOptionFlags(0)))
let data = CVPixelBufferGetBaseAddress(managedPixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGContext(data: data, width: Int(self.outputSize.width), height: Int(self.outputSize.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(managedPixelBuffer), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue)
context!.clear(CGRect(x: 0, y: 0, width: CGFloat(self.outputSize.width), height: CGFloat(self.outputSize.height)))
let horizontalRatio = CGFloat(self.outputSize.width) / nextPhoto.size.width
let verticalRatio = CGFloat(self.outputSize.height) / nextPhoto.size.height
//let aspectRatio = max(horizontalRatio, verticalRatio) // ScaleAspectFill
let aspectRatio = min(horizontalRatio, verticalRatio) // ScaleAspectFit
let newSize: CGSize = CGSize(width: nextPhoto.size.width * aspectRatio, height: nextPhoto.size.height * aspectRatio)
let x = newSize.width < self.outputSize.width ? (self.outputSize.width - newSize.width) / 2 : 0
let y = newSize.height < self.outputSize.height ? (self.outputSize.height - newSize.height) / 2 : 0
context?.draw(nextPhoto.cgImage!, in: CGRect(x: x, y: y, width: newSize.width, height: newSize.height))
CVPixelBufferUnlockBaseAddress(managedPixelBuffer, CVPixelBufferLockFlags(rawValue: CVOptionFlags(0)))
appendSucceeded = pixelBufferAdaptor.append(pixelBuffer, withPresentationTime: presentationTime)
} else {
print("Failed to allocate pixel buffer")
appendSucceeded = false
}
}
if !appendSucceeded {
break
}
frameCount += 1
}
videoWriterInput.markAsFinished()
videoWriter.finishWriting { () -> Void in
print("-----video1 url = \(self.imageArrayToVideoURL)")
self.globalVideoURL = self.imageArrayToVideoURL
self.asset = AVAsset.init(url:self.imageArrayToVideoURL as URL)
self.exportVideoWithAnimation()
}
})
}
}
3)Added animations to videos.
func exportVideoWithAnimation() {
let composition = AVMutableComposition()
let track = self.asset?.tracks(withMediaType: AVMediaType.video)
let videoTrack:AVAssetTrack = track![0] as AVAssetTrack
let timerange = CMTimeRangeMake(kCMTimeZero, (self.asset?.duration)!)
let compositionVideoTrack:AVMutableCompositionTrack = composition.addMutableTrack(withMediaType: AVMediaType.video, preferredTrackID: CMPersistentTrackID())!
do {
try compositionVideoTrack.insertTimeRange(timerange, of: videoTrack, at: kCMTimeZero)
compositionVideoTrack.preferredTransform = videoTrack.preferredTransform
} catch {
print(error)
}
//if your video has sound, you don’t need to check this
if self.audioIsEnabled {
let compositionAudioTrack:AVMutableCompositionTrack = composition.addMutableTrack(withMediaType: AVMediaType.audio, preferredTrackID: CMPersistentTrackID())!
for audioTrack in (self.asset?.tracks(withMediaType: AVMediaType.audio))! {
do {
try compositionAudioTrack.insertTimeRange(audioTrack.timeRange, of: audioTrack, at: kCMTimeZero)
} catch {
print(error)
}
}
}
let size = videoTrack.naturalSize
let videolayer = CALayer()
videolayer.frame = CGRect(x: 0, y: 0, width: size.width, height: size.height)
let parentlayer = CALayer()
parentlayer.frame = CGRect(x: 0, y: 0, width: size.width, height: size.height)
parentlayer.addSublayer(videolayer)
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//this is the animation part
var time = [0.00001, 3, 6, 9, 12] //I used this time array to determine the start time of a frame animation. Each frame will stay for 3 secs, thats why their difference is 3
var imgarray = self.selectedImageArray
for image in 0..<self.selectedImageArray.count {
let nextPhoto = imgarray[image]
let horizontalRatio = CGFloat(self.outputSize.width) / nextPhoto.size.width
let verticalRatio = CGFloat(self.outputSize.height) / nextPhoto.size.height
let aspectRatio = min(horizontalRatio, verticalRatio)
let newSize: CGSize = CGSize(width: nextPhoto.size.width * aspectRatio, height: nextPhoto.size.height * aspectRatio)
let x = newSize.width < self.outputSize.width ? (self.outputSize.width - newSize.width) / 2 : 0
let y = newSize.height < self.outputSize.height ? (self.outputSize.height - newSize.height) / 2 : 0
let blackLayer = CALayer()
///#7. opacity(1->0)(top->bottom)///
//#3. top->bottom///
//MARK:- Animations==================================
///#1. left->right///
if(self.globalSelectedTransitionTag == 0){
blackLayer.frame = CGRect(x: -videoTrack.naturalSize.width, y: 0, width: videoTrack.naturalSize.width, height: videoTrack.naturalSize.height)
blackLayer.backgroundColor = UIColor.black.cgColor
let imageLayer = CALayer()
imageLayer.frame = CGRect(x: x, y: y, width: newSize.width, height: newSize.height)
imageLayer.contents = imgarray[image].cgImage
blackLayer.addSublayer(imageLayer)
let animation = CABasicAnimation()
animation.keyPath = "position.x"
animation.fromValue = -videoTrack.naturalSize.width
animation.toValue = 5 * (videoTrack.naturalSize.width)
animation.duration = 5
animation.beginTime = CFTimeInterval(time[image])
animation.fillMode = kCAFillModeForwards
animation.isRemovedOnCompletion = false
blackLayer.add(animation, forKey: "opacity")
}
parentlayer.addSublayer(blackLayer)
}
let layercomposition = AVMutableVideoComposition()
layercomposition.frameDuration = CMTimeMake(1, 30)
layercomposition.renderSize = size
layercomposition.animationTool = AVVideoCompositionCoreAnimationTool(postProcessingAsVideoLayer: videolayer, in: parentlayer)
let instruction = AVMutableVideoCompositionInstruction()
instruction.timeRange = CMTimeRangeMake(kCMTimeZero, composition.duration)
let videotrack = composition.tracks(withMediaType: AVMediaType.video)[0] as AVAssetTrack
let layerinstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: videotrack)
instruction.layerInstructions = [layerinstruction]
layercomposition.instructions = [instruction]
if(fromTransition){
self.globalrVideoComposition = layercomposition
}
let animatedVideoURL = NSURL(fileURLWithPath: NSHomeDirectory() + "/Documents/video2.mp4")
self.removeFileAtURLIfExists(url: animatedVideoURL)
guard let assetExport = AVAssetExportSession(asset: composition, presetName:AVAssetExportPresetHighestQuality) else {return}
assetExport.videoComposition = self.globalrVideoComposition
assetExport.outputFileType = AVFileType.mp4
assetExport.outputURL = animatedVideoURL as URL
print("****** animatedVideoURL *****",animatedVideoURL)
assetExport.exportAsynchronously(completionHandler: {
switch assetExport.status{
case AVAssetExportSessionStatus.failed:
print("failed \(String(describing: assetExport.error))")
case AVAssetExportSessionStatus.cancelled:
print("cancelled \(String(describing: assetExport.error))")
default:
print("Exported")
if(self.fromPlayVideo){
DispatchQueue.main.async {
self.globalVideoURL = animatedVideoURL; self.playVideoInPlayer(animatedVideoURL: animatedVideoURL as URL)
}
}else if(self.fromSave){
PHPhotoLibrary.shared().performChanges({
PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: animatedVideoURL as URL)
print("222222 animatedVideoURL",animatedVideoURL)
}) { saved, error in
DispatchQueue.main.async {
MBProgressHUD.hideAllHUDs(for: self.view, animated: true)
}
if saved {
let alertController = UIAlertController(title: "Your video was successfully saved", message: nil, preferredStyle: .alert)
let defaultAction = UIAlertAction(title: "OK", style: .default, handler: nil)
alertController.addAction(defaultAction)
print("The task is done,enjoy now!")
self.present(alertController, animated: true, completion: nil)
}else{
}
}
}
}
})
}
4)Played Videos
func playVideoInPlayer(animatedVideoURL:URL){
if(globalFilterName != nil){
self.asset = AVAsset.init(url:animatedVideoURL as URL)
let newPlayerItem = AVPlayerItem.init(asset:self.asset);
newPlayerItem.videoComposition=globalrVideoComposition
self.player = AVPlayer.init(playerItem:newPlayerItem)
}else{
let newPlayerItem = AVPlayerItem.init(url:animatedVideoURL)
self.player = AVPlayer.init(playerItem:newPlayerItem)
}
NotificationCenter.default.addObserver(self, selector: #selector(self.finishedPlaying(_:)), name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object:nil)
self.playerLayer = AVPlayerLayer.init(player:self.player)
let width: CGFloat = self.videoContainerView.frame.size.width
let height: CGFloat = self.videoContainerView.frame.size.height
self.playerLayer.frame = CGRect(x: 0.0, y:0, width: width, height: height)
self.playerLayer.backgroundColor = UIColor.black.cgColor
self.playerLayer.videoGravity = .resizeAspectFill
self.videoContainerView.layer.addSublayer( self.playerLayer)
self.playPauseBtn.isHidden = false
self.playPauseBtn.setImage(UIImage.init(named:"pause"), for:.normal)
DispatchQueue.main.async {
MBProgressHUD.hideAllHUDs(for:self.view, animated:true)
self.player.play()
}
}
This whole task is woking fine the only problem is ,it is taking a lot of time to play the video with all setups(conversion image to video and adding animations).
Please help me to reduce the time so that the user has to not wait for long time when they land to play video after collecting images from imagepicker.
Any help or guidance would be highly appreciated.Thanks in advance!

What is wrong with my swift VideoWriter code?

I want to generate a mpeg video from a few images (as frames).
After I finish writing the video file, I tried saving it to Photos, but iOS consider it incompatible:
let compatible = AVAsset(url: video_url).isCompatibleWithSavedPhotosAlbum
print("COMPATIBILITY", compatible) // false
And then I tried creating an AVPlayer to play the video, and the video fails to play. So the video file must be corrupt somehow.
I reviewed my code closely, but couldn't spot the problem. Please help.
Here is my code:
class VideoWriter {
var url:URL?
var assetWriter:AVAssetWriter?
init(url:URL) {
self.url = url
do {
try self.assetWriter = AVAssetWriter(url: self.url!, fileType: AVFileTypeMPEG4)
} catch {
print("Fail to create assetWriter")
}
}
func writeFrames(frames:[UIImage], finishedHandler:#escaping ()->Void) {
let settings:[String:Any] = [
AVVideoCodecKey: AVVideoCodecH264,
AVVideoWidthKey: 480, //CANVAS_SIZE * 4 / 3,
AVVideoHeightKey: 360 //CANVAS_SIZE
]
let assetWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: settings)
self.assetWriter?.add(assetWriterInput)
let bufferAttributes:[String: Any] = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32ARGB)]
let bufferAdapter = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: assetWriterInput, sourcePixelBufferAttributes: bufferAttributes)
let frameTime = CMTimeMake(1, 30)
self.assetWriter?.startWriting()
self.assetWriter?.startSession(atSourceTime: kCMTimeZero)
// write the frames here
let mediaInputQueue = DispatchQueue(label: "mediaInputQueue")
var i = 0
let frameNumber = frames.count
assetWriterInput.requestMediaDataWhenReady(on: mediaInputQueue){
while(true){
if(i >= frameNumber){
break
}
if (assetWriterInput.isReadyForMoreMediaData){
let image = frames[i]
print("writing frame ", i)
let pixelBuffer = self.newPixelBufferFrom(cgImage: image.cgImage!)
var time:CMTime
if i == 0 {
time = kCMTimeZero
} else {
let value = i - 1
let lastTime = CMTimeMake(Int64(value), frameTime.timescale)
time = CMTimeAdd(lastTime, frameTime)
}
bufferAdapter.append(pixelBuffer!, withPresentationTime: time)
i += 1
}
}
assetWriterInput.markAsFinished()
self.assetWriter?.finishWriting(completionHandler: {
Thread.sleep(forTimeInterval: 0.5)
DispatchQueue.main.sync {
print("Completed?", self.assetWriter?.status == AVAssetWriterStatus.completed)
finishedHandler()
}
})
}
}
func newPixelBufferFrom(cgImage:CGImage) -> CVPixelBuffer?{
let options:[String: Any] = [kCVPixelBufferCGImageCompatibilityKey as String: true, kCVPixelBufferCGBitmapContextCompatibilityKey as String: true]
var pxbuffer:CVPixelBuffer?
let frameWidth = 480 //CANVAS_SIZE
let frameHeight = 360 //CANVAS_SIZE
let status = CVPixelBufferCreate(kCFAllocatorDefault, frameWidth, frameHeight, kCVPixelFormatType_32ARGB, options as CFDictionary?, &pxbuffer)
// TODO: throw exception in case of error, don't use assert
assert(status == kCVReturnSuccess && pxbuffer != nil, "newPixelBuffer failed")
CVPixelBufferLockBaseAddress(pxbuffer!, CVPixelBufferLockFlags(rawValue: 0))
let pxdata = CVPixelBufferGetBaseAddress(pxbuffer!)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGContext(data: pxdata, width: frameWidth, height: frameHeight, bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pxbuffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)
// TODO: throw exception in case of error, don't use assert
assert(context != nil, "context is nil")
context!.concatenate(CGAffineTransform.identity)
context!.draw(cgImage, in: CGRect(x: 0, y: 0, width: cgImage.width, height: cgImage.height))
CVPixelBufferUnlockBaseAddress(pxbuffer!, CVPixelBufferLockFlags(rawValue: 0))
return pxbuffer
}
}
btw. I didn't add audio input, is that necessary for a MPEG file?

iOS swift memory issue during a while-loop inside a closure: AVAssetWriter

My app is for making a video file from many images that are produced by code.
When my code has finished making an image and put it in myImage, it toggles isImageReady to 'true'. And when self.i is set(or changed),by Property Observer,it starts making another image. and finally the self.iReset is set to 'true' when there's no more image to be produced.
but the app is terminated due to memory issue during the while-loop. I have commented out the if-statement that actually assembles video frames. and it still has a memory issue. so I think the poblem lives during the while-loop inside requestMediaDataWhenReadyOnQueue:usingBlock closure.
I have no idea how to solve the problem. please help me.
if videoWriter.startWriting() {
videoWriter.startSessionAtSourceTime(kCMTimeZero)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = dispatch_queue_create("mediaInputQueue", nil)
videoWriterInput.requestMediaDataWhenReadyOnQueue(media_queue, usingBlock: { () -> Void in
let fps: Int32 = 30
let frameDuration = CMTimeMake(1, fps)
var lastFrameTime:CMTime = CMTime()
var presentationTime:CMTime = CMTime()
while (self.iReset != true) {
if videoWriterInput.readyForMoreMediaData && self.isImageReady {
lastFrameTime = CMTimeMake(Int64(self.i), fps)
presentationTime = self.i == 1 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
//commented out for tracking
/* if !self.appendPixelBufferForImage(self.myImage, pixelBufferAdaptor: pixelBufferAdaptor, presentationTime: presentationTime) {
error = NSError(
domain: kErrorDomain,
code: kFailedToAppendPixelBufferError,
userInfo: [
"description": "AVAssetWriterInputPixelBufferAdapter failed to append pixel buffer",
"rawError": videoWriter.error ?? "(none)"])
break
} */
self.isImageReady = false
self.i++
}// if ..&&..
} //while loop ends
// Finish writing
videoWriterInput.markAsFinished()
videoWriter.finishWritingWithCompletionHandler { () -> Void in
if error == nil {
print("Finished Making a Movie !!")
success(videoOutputURL)
}
self.videoWriter = nil
}
}) // requestMediaDataWhenReadyOnQueue ends
}
Probably too late but I had a similar issue to this (images in a loop) which caused me a major headache. My solution was to put an autoreleasepool within the loop which solved the issue.
As per #C. Carter has suggested you can use autoreleasepool to free the used memory like below:
autoreleasepool {
/* your code */
}
Other than that here is a code from which I am making movie using UIImages and Audio which works perfectly fine.
func build(chosenPhotos: [UIImage], audioURL: NSURL, completion: (NSURL) -> ()) {
showLoadingIndicator(appDel.window!.rootViewController!.view)
let outputSize = CGSizeMake(640, 480)
var choosenPhotos = chosenPhotos
let fileManager = NSFileManager.defaultManager()
let urls = fileManager.URLsForDirectory(.DocumentDirectory, inDomains: .UserDomainMask)
guard let documentDirectory: NSURL = urls.first else {
fatalError("documentDir Error")
}
let videoOutputURL = documentDirectory.URLByAppendingPathComponent("OutputVideo.mp4")
print("Video Output URL \(videoOutputURL)")
if NSFileManager.defaultManager().fileExistsAtPath(videoOutputURL.path!) {
do {
try NSFileManager.defaultManager().removeItemAtPath(videoOutputURL.path!)
} catch {
fatalError("Unable to delete file: \(error) : \(#function).")
}
}
guard let videoWriter = try? AVAssetWriter(URL: videoOutputURL, fileType: AVFileTypeMPEG4) else {
fatalError("AVAssetWriter error")
}
let outputSettings = [AVVideoCodecKey : AVVideoCodecH264, AVVideoWidthKey : NSNumber(float: Float(outputSize.width)), AVVideoHeightKey : NSNumber(float: Float(outputSize.height))]
guard videoWriter.canApplyOutputSettings(outputSettings, forMediaType: AVMediaTypeVideo) else {
fatalError("Negative : Can't apply the Output settings...")
}
let videoWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: outputSettings)
let sourcePixelBufferAttributesDictionary = [kCVPixelBufferPixelFormatTypeKey as String : NSNumber(unsignedInt: kCVPixelFormatType_32ARGB), kCVPixelBufferWidthKey as String: NSNumber(float: Float(outputSize.width)), kCVPixelBufferHeightKey as String: NSNumber(float: Float(outputSize.height))]
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: videoWriterInput, sourcePixelBufferAttributes: sourcePixelBufferAttributesDictionary)
if videoWriter.canAddInput(videoWriterInput) {
videoWriter.addInput(videoWriterInput)
}
if videoWriter.startWriting() {
videoWriter.startSessionAtSourceTime(kCMTimeZero)
assert(pixelBufferAdaptor.pixelBufferPool != nil)
let media_queue = dispatch_queue_create("mediaInputQueue", nil)
videoWriterInput.requestMediaDataWhenReadyOnQueue(media_queue, usingBlock: { () -> Void in
let fps: Int32 = 1
let frameDuration = CMTimeMake(1, fps)
var frameCount: Int64 = 0
var appendSucceeded = true
while (!choosenPhotos.isEmpty) {
if (videoWriterInput.readyForMoreMediaData) {
let nextPhoto = choosenPhotos.removeAtIndex(0)
let lastFrameTime = CMTimeMake(frameCount, fps)
let presentationTime = frameCount == 0 ? lastFrameTime : CMTimeAdd(lastFrameTime, frameDuration)
var pixelBuffer: CVPixelBuffer? = nil
let status: CVReturn = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pixelBufferAdaptor.pixelBufferPool!, &pixelBuffer)
if let pixelBuffer = pixelBuffer where status == 0 {
let managedPixelBuffer = pixelBuffer
CVPixelBufferLockBaseAddress(managedPixelBuffer, 0)
let data = CVPixelBufferGetBaseAddress(managedPixelBuffer)
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let context = CGBitmapContextCreate(data, Int(outputSize.width), Int(outputSize.height), 8, CVPixelBufferGetBytesPerRow(managedPixelBuffer), rgbColorSpace, CGImageAlphaInfo.PremultipliedFirst.rawValue)
CGContextClearRect(context, CGRectMake(0, 0, CGFloat(outputSize.width), CGFloat(outputSize.height)))
let horizontalRatio = CGFloat(outputSize.width) / nextPhoto.size.width
let verticalRatio = CGFloat(outputSize.height) / nextPhoto.size.height
//aspectRatio = max(horizontalRatio, verticalRatio) // ScaleAspectFill
let aspectRatio = min(horizontalRatio, verticalRatio) // ScaleAspectFit
let newSize:CGSize = CGSizeMake(nextPhoto.size.width * aspectRatio, nextPhoto.size.height * aspectRatio)
let x = newSize.width < outputSize.width ? (outputSize.width - newSize.width) / 2 : 0
let y = newSize.height < outputSize.height ? (outputSize.height - newSize.height) / 2 : 0
CGContextDrawImage(context, CGRectMake(x, y, newSize.width, newSize.height), nextPhoto.CGImage)
CVPixelBufferUnlockBaseAddress(managedPixelBuffer, 0)
appendSucceeded = pixelBufferAdaptor.appendPixelBuffer(pixelBuffer, withPresentationTime: presentationTime)
} else {
print("Failed to allocate pixel buffer")
appendSucceeded = false
}
}
if !appendSucceeded {
break
}
frameCount += 1
}
videoWriterInput.markAsFinished()
videoWriter.finishWritingWithCompletionHandler { () -> Void in
print("FINISHED!!!!!")
self.compileToMakeMovie(videoOutputURL, audioOutPutURL: audioURL, completion: { url in
completion(url)
})
}
})
}
}
func compileToMakeMovie(videoOutputURL: NSURL, audioOutPutURL: NSURL, completion: (NSURL) -> ()){
let mixComposition = AVMutableComposition()
let fileManager = NSFileManager.defaultManager()
let urls = fileManager.URLsForDirectory(.DocumentDirectory, inDomains: .UserDomainMask)
guard let documentDirectory: NSURL = urls.first else {
fatalError("documentDir Error")
}
let actualVideoURl = documentDirectory.URLByAppendingPathComponent("OutputVideoMusic.mp4")
print("Video Output URL \(actualVideoURl)")
if NSFileManager.defaultManager().fileExistsAtPath(actualVideoURl.path!) {
do {
try NSFileManager.defaultManager().removeItemAtPath(actualVideoURl.path!)
} catch {
fatalError("Unable to delete file: \(error) : \(#function).")
}
}
let nextClipStartTime = kCMTimeZero
let videoAsset = AVURLAsset(URL: videoOutputURL)
let video_timeRange = CMTimeRangeMake(kCMTimeZero,videoAsset.duration)
let a_compositionVideoTrack = mixComposition.addMutableTrackWithMediaType(AVMediaTypeVideo, preferredTrackID: kCMPersistentTrackID_Invalid)
try! a_compositionVideoTrack.insertTimeRange(video_timeRange, ofTrack: videoAsset.tracksWithMediaType(AVMediaTypeVideo)[0], atTime: nextClipStartTime)
let audioAsset = AVURLAsset(URL: audioOutPutURL)
let audio_timeRange = CMTimeRangeMake(kCMTimeZero,audioAsset.duration)
let b_compositionAudioTrack = mixComposition.addMutableTrackWithMediaType(AVMediaTypeAudio, preferredTrackID: kCMPersistentTrackID_Invalid)
do {
try b_compositionAudioTrack.insertTimeRange(audio_timeRange, ofTrack: audioAsset.tracksWithMediaType(AVMediaTypeAudio)[0], atTime: nextClipStartTime)
}catch _ {}
let assetExport = AVAssetExportSession.init(asset: mixComposition, presetName: AVAssetExportPresetHighestQuality)
assetExport?.outputFileType = "com.apple.quicktime-movie"
assetExport?.outputURL = actualVideoURl
assetExport?.exportAsynchronouslyWithCompletionHandler({
completion(actualVideoURl)
})
}
Let me know, if you're still facing the issue.

Pixel Buffer will not append to AVAssetWriterInput on device

I'm trying to take an image and convert it into a video file in Swift. I was able to get the code to work in the iOS simulator, but it won't work on an actual device. It errors out when I try to append the pixel buffer to the AVAssetWriterInput. I was wondering if anyone here has run into the same issue previously. Here's the code:
func createVideoFromImage(img: CGImageRef) -> Void {
var error: NSError?
let frameSize = CGSizeMake(CGFloat(CGImageGetWidth(img)), CGFloat(CGImageGetHeight(img)))
let fileName = "\(uniqueString()).m4v"
let assetWriter = AVAssetWriter(URL: fileUrl(fileName), fileType: AVFileTypeAppleM4V, error: &error)
let outputSettings = [
AVVideoCodecKey: AVVideoCodecH264,
AVVideoWidthKey: frameSize.width,
AVVideoHeightKey: frameSize.height
]
let assetWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: outputSettings)
let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: assetWriterInput, sourcePixelBufferAttributes: nil)
if assetWriter.canAddInput(assetWriterInput) {
assetWriter.addInput(assetWriterInput)
}
assetWriter.startWriting()
assetWriter.startSessionAtSourceTime(kCMTimeZero)
let buffer:CVPixelBufferRef = pixelBufferFromCGImage(img, frameSize: frameSize)
if pixelBufferAdaptor.assetWriterInput.readyForMoreMediaData {
let frameTime = CMTimeMakeWithSeconds(0, 30)
let pixelBufferAppend = pixelBufferAdaptor.appendPixelBuffer(buffer, withPresentationTime: frameTime)
if pixelBufferAppend {
assetWriterInput.markAsFinished()
assetWriter.finishWritingWithCompletionHandler { () -> Void in
switch assetWriter.status {
case AVAssetWriterStatus.Failed:
println("Error: \(assetWriter.error)")
default:
let path = self.fileUrl(fileName).path!
let content = NSFileManager.defaultManager().contentsAtPath(path)
println("Video: \(path) \(content?.length)")
}
}
} else {
println("failed to append pixel buffer")
}
}
}
func pixelBufferFromCGImage (img: CGImageRef, frameSize: CGSize) -> CVPixelBufferRef {
let options = [
"kCVPixelBufferCGImageCompatibilityKey": true,
"kCVPixelBufferCGBitmapContextCompatibilityKey": true
]
var pixelBufferPointer = UnsafeMutablePointer<Unmanaged<CVPixelBuffer>?>.alloc(1)
let buffered:CVReturn = CVPixelBufferCreate(kCFAllocatorDefault, UInt(frameSize.width), UInt(frameSize.height), OSType(kCVPixelFormatType_32ARGB), options, pixelBufferPointer)
let lockBaseAddress = CVPixelBufferLockBaseAddress(pixelBufferPointer.memory?.takeUnretainedValue(), 0)
var pixelData:UnsafeMutablePointer<(Void)> = CVPixelBufferGetBaseAddress(pixelBufferPointer.memory?.takeUnretainedValue())
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.NoneSkipFirst.rawValue)
let space:CGColorSpace = CGColorSpaceCreateDeviceRGB()
var context:CGContextRef = CGBitmapContextCreate(pixelData, UInt(frameSize.width), UInt(frameSize.height), 8, CVPixelBufferGetBytesPerRow(pixelBufferPointer.memory?.takeUnretainedValue()), space, bitmapInfo)
CGContextDrawImage(context, CGRectMake(0, 0, frameSize.width, frameSize.height), img)
CVPixelBufferUnlockBaseAddress(pixelBufferPointer.memory?.takeUnretainedValue(), 0)
return pixelBufferPointer.memory!.takeUnretainedValue()
}
pixelBufferAppend returns false! Any help would be greatly appreciated. I have been scratching my head on this all night!
I was finally able to figure what was going on. The device could only support encoding frames up to 1920x1080px - reduced the image size and it worked. Change frameSize to be a variable and use the following code:
if frameSize.width > maxImageSize.width || frameSize.height > maxImageSize.height {
let scale = min((maxImageSize.width/frameSize.width), (maxImageSize.height/frameSize.height))
frameSize = CGSizeMake((scale * frameSize.width), (scale * frameSize.height))
}

Resources