I'm currently trying to put 5 videos back to back using AVMutableComposition like so:
[mixComposition insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset1.duration) ofAsset:asset1 atTime:[mixComposition duration] error:nil];
[mixComposition insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset2.duration) ofAsset:asset2 atTime:[mixComposition duration] error:nil];
[mixComposition insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset3.duration) ofAsset:asset3 atTime:[mixComposition duration] error:nil];
[mixComposition insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset4.duration) ofAsset:asset4 atTime:[mixComposition duration] error:nil];
[mixComposition insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset5.duration) ofAsset:asset5 atTime:[mixComposition duration] error:nil];
I then use an AVAssetExportSession to export the video, which works however between each video I'm getting a blank/black frame which I need to remove. Has anyone had this problem before and if so did you manage to fix it?
Also, the blank frames aren't in the source video files.
Thanks in advance.
I had the same problem the other day.
If you got your assets(asset1, asset2, etc.) by exporting them from other asset which was created using again insertTimeRange, then it's the same case.
The problem is that when you export assets created using insertTimeRange, the export doesn't go correct, when you attach such videos black frames appear between them.
Try using "timeRange" option of the AVAssetExportSession, and cut the range you need from the main asset. Then your assets will attach correct.
I just met the same problem. Turn out the solution is using AVMutableCompositionTrack and compositing the video tracks, something like this.
AVMutableCompositionTrack * videoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[videoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset.duration) ofTrack:[[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:composition.duration error:&error];
For the audio part I think you must add a separate track with AVMediaTypeAudio type.
Related
have been working on the AVMutableComposition to mix audio file with video,
For the part of insert the audio at video time 0, am using this
AVMutableCompositionTrack *a_compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[a_compositionVideoTrack insertTimeRange:video_timeRange ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:kCMTimeZero error:nil];
My challenge now is to let the user pick the video time range he want the audio in !! have no idea how this works with the CMTimeMake and if there is any smoothy picker already done
Thanks for helping !!
CMTimeMake(value,timescale)
value - as usual, amount of quantums (for example, seconds)
timescale - length of this quantum in seconds
CMTimeMake(1,30) // one interval of 30 sec
CMTimeMake(30,1) // 30 intervals of 1 sec
In fact it is the same absolute time, But it has different granularity, which is important when you deal with audio and video file processing.
I'm trying to create an AVMutableComposition and playing it using AVPlayer. This is what I'm doing.
//Define the AVComposition
self.composition = [[AVMutableComposition alloc] init];
//Define Mutable Track
self.mainVideoTrack = [self.composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
//Define the asset
AVAsset *asset = [AVAsset assetWithURL:url];
//Insert the asset in the track
[self.mainVideoTrack insertTimeRange:CMTimeRangeFromTimeToTime(CMTimeMake(0,1000),CMTimeMake(asset.duration*1000,1000)) ofTrack:[[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:[self startingTimeForSegment:videoSegment] error:&error];
//Define player item with the composition
self.playerItem = [[AVPlayerItem alloc] initWithAsset:self.composition];
//Define the player, player layer & add the layer
self.avPlayer = [[AVPlayer alloc]initWithPlayerItem:self.playerItem];
self.layerVideo = [AVPlayerLayer playerLayerWithPlayer:self.avPlayer];
self.layerVideo.frame = self.previewContainer.bounds;
[self.previewContainer.layer addSublayer:self.layerVideo];
However I can hear no sound in the player layer. If I initialize the playeritem directly with the asset without using AVMutableComposition, it plays the sound. What am I doing wrong? Stuck on this.
It looks like you are only adding one track to your AVMutableComposition. You would need to add the Audio track from the asset as well. This can be done exactly the same way as adding a video track.
self.mainAudioTrack = [self.composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
And then you can insert from your assets audio track. This is assuming you want your audio to start at the same point your video does.
[self.mainAudioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,asset.duration) ofTrack:[[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:[self startingTimeForSegment:audio] error:&error];
We have a video player where we play videos inside an AVPlayer (1GB of content in about 8MB .mov files in size). We load the AVPlayer using an AVMutableComposition of a video track and audio track that are on local disk bundled with the app.
We do something like:
AVAsset* videoAsset = [[AVURLAsset alloc] initWithURL:videoUrl options:nil];
AVAsset* voiceAsset = useVoice ? [[AVURLAsset alloc] initWithURL:voiceUrl options:nil] : nil;
AVMutableComposition* composition = [[AVMutableComposition alloc] init];
AVMutableCompositionTrack* videoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVMutableCompositionTrack* audioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
AVMutableCompositionTrack* voiceTrack = useVoice ? [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid] : nil;
NSError* error = nil;
[videoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] firstObject] atTime:kCMTimeZero error:&error];
if (error) {
[[MNGAppDelegate sharedManagers].errorManager presentError:error];
}
if ([videoAsset tracksWithMediaType:AVMediaTypeAudio].count > 0) {
[audioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeAudio] firstObject] atTime:kCMTimeZero error:&error];
if (error) {
[[MNGAppDelegate sharedManagers].errorManager presentError:error];
}
}
if (useVoice) {
[voiceTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, voiceAsset.duration) ofTrack:[[voiceAsset tracksWithMediaType:AVMediaTypeAudio] firstObject] atTime:kCMTimeZero error:&error];
if (error) {
[[MNGAppDelegate sharedManagers].errorManager presentError:error];
}
}
And we load it using a replaceCurrentItemWithPlayerItem (except for the first one).
[self.player replaceCurrentItemWithPlayerItem:nextItem];
We never create a playlist or can go back. We simply replace it when a new video needs to be played.
What we're noticing is that VM Tracker is showing our Dirty Size is going crazy. Once we play the first 8MB file we approach about 80MB of dirty. As we replace more and more videos we can easily get our Dirty Size to 200MB+. Within about 20-30 videos the app will usually be killed and we get a Low Memory Crash Log.
Is there something special we should be doing to reduce the memory of AVPlayer as we replace clips in the player?
I have found that setting:
[someAssetWriterInput setExpectsMediaDataInRealTime:NO];
.. has some effect on memory pressure experienced during AVComposition-oriented export sessions .. it would appear to be at least one way to govern the internal memory usage within the framework ..
self.player?,pause()
self.activityIndicator?.startAnimating()
self.removeObserverPlayerItem()
let item = AVPlayerItem(url: fileurl)
player?.replaceCurrentItem(with: item)
self.addObserverPlayerItem()
self.player?.play()
This will controll your memory and will take only needed memory. this resolved my problem.
This question is quite related to AVMutableComposition - Blank/Black frame between videos assets but as I am not using an AVAssetExportSession the answers doesn't fit my problem.
I'm using an AVMutableComposition to create a video composition and I'm reading it using an AVAssetReader (I need to have the frame data, I can't use a AVPlayer) but I often have black frames between my video chunks (there is no glitch noticeable in the audio).
I create my Composition as
AVMutableCompositionTrack *compositionVideoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVMutableCompositionTrack *compositionAudioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
NSMutableArray* durationList = [NSMutableArray array];
NSMutableArray* videoList= [NSMutableArray array];
NSMutableArray* audioList= [NSMutableArray array];
for (NSInteger i = 0; i < [clips count]; i++)
{
AVURLAsset *myasset = [clips objectAtIndex:i];
AVAssetTrack *clipVideoTrack = [[myasset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
[videoList addObject:clipVideoTrack];
AVAssetTrack *clipAudioTrack = [[myasset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
[audioList addObject:clipAudioTrack];
CMTime clipDuration = [myasset duration];
CMTimeRange clipRange = CMTimeRangeMake(kCMTimeZero, clipDuration);
[durationList addObject:[NSValue valueWithCMTimeRange:clipRange]];
}
[compositionVideoTrack insertTimeRanges:durationList ofTracks:videoList atTime:kCMTimeZero error:nil];
[compositionAudioTrack insertTimeRanges:durationList ofTracks:audioList atTime:kCMTimeZero error:nil];
I also tried to insert each track manually in my composition but I have the same phenomenon.
Thanks
I managed to solve this problem after many hours of testing. There are two things I needed to change.
1) add the 'precise' option when creating the asset.
NSDictionary *options = #{AVURLAssetPreferPreciseDurationAndTimingKey:#YES};
AVURLAsset *videoAsset3 = [AVURLAsset URLAssetWithURL:clip3URL options:options];
2) don't use
CMTime duration3 = [videoAsset3 duration];
use instead
AVAssetTrack *videoAsset3Track = [[videoAsset3 tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
CMTime duration3 = videoAsset3Track.timeRange.duration;
I found this out after I set the AVPlayer background color to Blue, then I noticed blue frames appearing, so the problem was to do with timings. Once I changed to above settings, the different videos aligned up fine when using:
AVMutableCompositionTrack *videoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo
preferredTrackID:kCMPersistentTrackID_Invalid];
[videoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, duration3)
ofTrack:videoAsset3Track
atTime:kCMTimeZero error:&error];
Try creating two video tracks, and then alternate between the two when adding the clips.
calculate Last CMTime is very important to insertTimeRange: for new AVAsset
I had the same problem and solve it with:
func addChunk(media: AVAsset) throws {
let duration = self.mutableComposition.tracks.last?.timeRange.end
try mutableComposition.insertTimeRange(CMTimeRange(start: .zero, duration: media.duration), of: media, at: duration ?? .zero)
}
I want to show a display overlay over a video and export that video including this display. I had a look into the AVFoundation Framework, AVCompositions, AVAssets etc. but I still do not have an idea to achieve this. There is a class called AVSynchronizedLayer which lets you animate things synchrounous to the video, but I do not want to animate, I jsut want to overlay the time display into every single frame of the video. Any advice?
Regards
Something like this...
(NB: culled from a much larger project, so I may have included some unnecessary pieces by accident).
You'll need to grab the CALayer of your clock / animation, and set it to the var myClockLayer (used 1/3 of the way down by the andimation tool).
This also assumes your incoming video has just two tracks - audio and video. If you have more, you'll need to set the track id in "asTrackID:2" more carefully.
AVURLAsset* url = [AVURLAsset URLAssetWithURL:incomingVideo options:nil];
AVMutableComposition *videoComposition = [AVMutableComposition composition];
AVMutableCompositionTrack *compositionVideoTrack = [videoComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVAssetTrack *clipVideoTrack = [[url tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
[compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, [url duration]) ofTrack:clipVideoTrack atTime:kCMTimeZero error:&error];
AVMutableVideoComposition* videoComposition = [[AVMutableVideoComposition videoComposition]retain];
videoComposition.renderSize = CGSizeMake(320, 240);
videoComposition.frameDuration = CMTimeMake(1, 30);
videoComposition.animationTool = [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithAdditionalLayer:myClockLayer asTrackID:2];
AVMutableVideoCompositionInstruction *instruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
instruction.timeRange = CMTimeRangeMake(kCMTimeZero, CMTimeMakeWithSeconds(60, 30) );
AVMutableVideoCompositionLayerInstruction* layerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:clipVideoTrack];
instruction.layerInstructions = [NSArray arrayWithObject:layerInstruction];
videoComposition.instructions = [NSArray arrayWithObject: instruction];
exporter = [[AVAssetExportSession alloc] initWithAsset:saveComposition presetName:AVAssetExportPresetHighestQuality] ;
exporter.videoComposition = videoComposition;
exporter.outputURL=url3;
exporter.outputFileType=AVFileTypeQuickTimeMovie;
[exporter exportAsynchronouslyWithCompletionHandler:^(void){}];
I think you can use AVCaptureVideoDataOutput to process each frame and use AVAssetWriter to record the processed frame.You can refer to this answer
https://stackoverflow.com/a/4944594/379941 .
use AVAssetWriterPixelBufferAdaptor's appendPixelBuffer:withPresentationTime: method to export
And I strongly suggest using OpenCV to process frame. this is a nice tutorial
http://aptogo.co.uk/2011/09/opencv-framework-for-ios/.
OpenCV library is very great.