compute times for cross-fade between videos - ios

Question

I have to apply opacity in a video. I have to apply it before the end of the video of a second. I am using the "firstInstruction" to have the total duration of the video. however when I call the "firstInstruction.setOpacityRamp" method I can not subtract the second one..
let mainInstruction = AVMutableVideoCompositionInstruction()
mainInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, CMTimeAdd(firstAsset.duration, secondAsset.duration))
let firstInstruction = VideoHelper.videoCompositionInstruction(firstTrack, asset: firstAsset)
firstInstruction.setOpacityRamp(fromStartOpacity: 1, toEndOpacity: 0.1, timeRange: mainInstruction.timeRange)

Answer 1

I would use three instructions to apply the cross-fade:
A “pass-through” instruction that shows only the first video track, until one second before the end of the first asset.
A cross-fade instruction that simultaneously shows the last second of the first video track and the first second of the second video track, with opacity ramps.
A “pass-through” instruction that shows only the second video track, starting from one second into the second video track.
So, first, let's get the tracks:
import AVFoundation
import CoreVideo
func crossFade(asset0: AVAsset, asset1: AVAsset, crossFadeDuration: CMTime, to outputUrl: URL) throws {
guard
let asset0Track = asset0.tracks(withMediaType: .video).first,
let asset1Track = asset1.tracks(withMediaType: .video).first,
case let composition = AVMutableComposition(),
case let compositionTrack0Id = composition.unusedTrackID(),
let compositionTrack0 = composition.addMutableTrack(
withMediaType: .video, preferredTrackID: compositionTrack0Id),
case let compositionTrack1Id = composition.unusedTrackID(),
let compositionTrack1 = composition.addMutableTrack(
withMediaType: .video, preferredTrackID: compositionTrack1Id)
else { return }
Now let's compute all of the times we need. First, the entire range of asset0Track in the composition, include both the pass-through and cross-fade periods:
// When does asset0Track start, in the composition?
let asset0TrackStartTime = CMTime.zero
// When does asset0Track end, in the composition?
let asset0TrackEndTime = asset0TrackStartTime + asset0Track.timeRange.duration
Next, the cross-fade's time range:
// When does the cross-fade end, in the composition?
// It should end exactly at the end of asset0's video track.
let crossFadeEndTime = asset0TrackEndTime
// When does the cross-fade start, in the composition?
let crossFadeStartTime = crossFadeEndTime - crossFadeDuration
// What is the entire time range of the cross-fade, in the composition?
let crossFadeTimeRange = CMTimeRangeMake(
start: crossFadeStartTime,
duration: crossFadeDuration)
Next, the entire range of asset1Track in the composition, include both the cross-fade and pass-through periods:
// When does asset1Track start, in the composition?
// It should start exactly at the start of the cross-fade.
let asset1TrackStartTime = crossFadeStartTime
// When does asset1Track end, in the composition?
let asset1TrackEndTime = asset1TrackStartTime + asset1Track.timeRange.duration
And finally, the two pass-through time ranges:
// What is the time range during which only asset0 is visible, in the composition?
let compositionTrack0PassThroughTimeRange = CMTimeRangeMake(
start: asset0TrackStartTime,
duration: crossFadeStartTime - asset0TrackStartTime)
// What is the time range during which only asset1 is visible, in the composition?
let compositionTrack1PassThroughTimeRange = CMTimeRangeMake(
start: crossFadeEndTime,
duration: asset1TrackEndTime - crossFadeEndTime)
Now we can insert the input tracks into the composition's tracks:
// Put asset0Track into compositionTrack0.
try compositionTrack0.insertTimeRange(
asset0Track.timeRange,of: asset0Track, at: asset0TrackStartTime)
// Put asset1Track into compositionTrack1.
try compositionTrack1.insertTimeRange(
asset1Track.timeRange, of: asset1Track, at: asset1TrackStartTime)
That is all we need to do for the AVMutableComposition. But we also need to make an AVMutableVideoComposition:
let videoComposition = AVMutableVideoComposition()
videoComposition.frameDuration =
min(asset0Track.minFrameDuration, asset1Track.minFrameDuration)
videoComposition.renderSize = CGSize(
width: max(asset0Track.naturalSize.width, asset1Track.naturalSize.width),
height: max(asset0Track.naturalSize.height, asset1Track.naturalSize.height))
We need to set the video composition's instructions. The first instruction is to pass through just compositionTrack0 for the appropriate time range:
// I'm using a helper function defined below.
let compositionTrack0PassThroughInstruction = AVMutableVideoCompositionInstruction.passThrough(
trackId: compositionTrack0Id, timeRange: compositionTrack0PassThroughTimeRange)
The second instruction is for the cross-fade, so it's more complicated. It needs two child instructions, one for each layer in the cross-fade. Each layer instruction, and the overall cross-fade instruction, use the same time range:
let crossFadeLayer0Instruction = AVMutableVideoCompositionLayerInstruction()
crossFadeLayer0Instruction.trackID = compositionTrack0Id
crossFadeLayer0Instruction.setOpacityRamp(fromStartOpacity: 1, toEndOpacity: 0, timeRange: crossFadeTimeRange)
let crossFadeLayer1Instruction = AVMutableVideoCompositionLayerInstruction()
crossFadeLayer1Instruction.trackID = compositionTrack1Id
crossFadeLayer1Instruction.setOpacityRamp(fromStartOpacity: 0, toEndOpacity: 1, timeRange: crossFadeTimeRange)
let crossFadeInstruction = AVMutableVideoCompositionInstruction()
crossFadeInstruction.timeRange = crossFadeTimeRange
crossFadeInstruction.layerInstructions = [crossFadeLayer0Instruction, crossFadeLayer1Instruction]
The third instruction is to pass through just compositionTrack1 for the appropriate time range:
let compositionTrack1PassThroughInstruction = AVMutableVideoCompositionInstruction.passThrough(
trackId: compositionTrack1Id, timeRange: compositionTrack1PassThroughTimeRange)
Now that we have all three instruction, we can give them to the video composition:
videoComposition.instructions = [compositionTrack0PassThroughInstruction, crossFadeInstruction, compositionTrack1PassThroughInstruction]
And now we can use composition and videoComposition together, for example to export a new movie file:
let export = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetMediumQuality)!
export.outputURL = outputUrl
export.videoComposition = videoComposition
export.exportAsynchronously {
exit(0)
}
}
Here's the helper I used to create the pass-through instructions:
extension AVMutableVideoCompositionInstruction {
static func passThrough(trackId: CMPersistentTrackID, timeRange: CMTimeRange) -> AVMutableVideoCompositionInstruction {
let layerInstruction = AVMutableVideoCompositionLayerInstruction()
layerInstruction.trackID = trackId
let instruction = AVMutableVideoCompositionInstruction()
instruction.timeRange = timeRange
instruction.layerInstructions = [layerInstruction]
return instruction
}
}
And here's my test code. I used a macOS command-line app for testing:
let asset0 = AVURLAsset(url: URL(fileURLWithPath: "/tmp/asset0.mp4"))
let asset1 = AVURLAsset(url: URL(fileURLWithPath: "/tmp/asset1.mp4"))
let outputUrl = URL(fileURLWithPath: "/tmp/output.mp4")
try! crossFade(asset0: asset0, asset1: asset1, crossFadeDuration: CMTimeMake(value: 1, timescale: 1), to: outputUrl)
dispatchMain()
Result:
Note that I had to make the animation tiny and low color because of Stack Overflow's limit on image file size.
Input videos courtesy of Jeffrey Beach.