CMSampleBuffer automatically rotate every time - ios

Question

Here I try to select video from photo library and read it frame by frame as a samplebuffer so that I can crop or rotate later.but the problem is CMSampleBuffer by default rotated.The variables are I use for initialization is
var asset:AVAsset! //load asset from url
var assetReader:AVAssetReader!
var assetVideoOutput: AVAssetReaderTrackOutput! // add assetreader for video
var assetAudioOutput: AVAssetReaderTrackOutput! // add assetreader for audio
var readQueue: DispatchQueue!
the settings of previous variables looks like this.
func resetRendering() {
do{
assetReader = try! AVAssetReader(asset: asset)
}catch {
print(error)
}
var tracks = asset.tracks(withMediaType: .video)
var track : AVAssetTrack!
if tracks.count > 0 {
track = tracks[0]
}
let decompressionvideoSettings:[String:Any] = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
assetVideoOutput = AVAssetReaderTrackOutput(track: track, outputSettings: decompressionvideoSettings)
tracks = asset.tracks(withMediaType: .audio)
if tracks.count > 0 {
track = tracks[0]
}
let audioReadSettings = [AVFormatIDKey: kAudioFormatLinearPCM]
assetAudioOutput = AVAssetReaderTrackOutput(track: track, outputSettings: audioReadSettings)
if (assetAudioOutput.responds(to: #selector(getter: AVAssetReaderOutput.alwaysCopiesSampleData))) {
assetAudioOutput.alwaysCopiesSampleData = false
}
if assetReader.canAdd(assetAudioOutput) {
assetReader.add(assetAudioOutput)
}
if (assetVideoOutput.responds(to: #selector(getter: AVAssetReaderOutput.alwaysCopiesSampleData))) {
assetVideoOutput.alwaysCopiesSampleData = false
}
if assetReader.canAdd(assetVideoOutput) {
assetReader.add(assetVideoOutput)
}
}
now when I try to read video frame by frame and convert into image it automatically rotate frame in 90 degrees.the conversion extention from samplebuffer to uiimage looks like this
extension CMSampleBuffer {
var uiImage: UIImage? {
guard let imageBuffer = CMSampleBufferGetImageBuffer(self) else { return nil }
CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags(rawValue: 0))
let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue)
guard let context = CGContext(data: baseAddress,
width: width,
height: height,
bitsPerComponent: 8,
bytesPerRow: bytesPerRow,
space: colorSpace,
bitmapInfo: bitmapInfo.rawValue) else { return nil }
guard let cgImage = context.makeImage() else { return nil }
CVPixelBufferUnlockBaseAddress(imageBuffer,CVPixelBufferLockFlags(rawValue: 0));
return UIImage(cgImage: cgImage)
}
}
resulted photo is
and actual photo is this one

Answer 1

Update
CGImage doesn't hold orientation property. So you have to manually set when you convert to UIImage. We get orientation from video track and set it to UIImage.
Below answer's right/left or up/down might be opposite. Pls try and check.
//pls get orientation property outside of image processing to avoid redundancy
let videoTrack = videoAsset.tracks(withMediaType: AVMediaType.video).first!
let videoSize = videoTrack.naturalSize
let transform = videoTrack.preferredTransform
var orientation = UIImage.Orientation.right
switch (transform.tx, transform.ty) {
case (0, 0): orientation = UIImage.Orientation.up
case (videoSize.width, videoSize.height): UIImage.Orientation.right
case (0, videoSize.width): UIImage.Orientation.left
default: orientation = UIImage.Orientation.down
}
// then set orientation in image processing such as captureOutput
let uiImage = UIImage.init(cgImage: cgImage, scale: 1.0, orientation: orientation)
Hope this helps.