GPUImageMovie not respecting imageOrientation - ios

I am using GPUImageMovie which is playing a movie file, this file was recorded on a iOS device.
However when GPUImageMovie plays it it is in the wrong orientation so the video isn't rotated to be shown correctly.
How do I get it to respect it's orientation ? I've tried modifying the OpenGL code with no luck.

I had same problem when playing a video recorded in iOS device using GPUImageMovie. I solved it using following functions :
Call this setRotationForFilter: method by passing you filter. The orientationForTrack: will return your current video orientation.
- (void)setRotationForFilter:(GPUImageOutput<GPUImageInput> *)filterRef {
UIInterfaceOrientation orientation = [self orientationForTrack:self.playerItem.asset];
if (orientation == UIInterfaceOrientationPortrait) {
[filterRef setInputRotation:kGPUImageRotateRight atIndex:0];
} else if (orientation == UIInterfaceOrientationLandscapeRight) {
[filterRef setInputRotation:kGPUImageRotate180 atIndex:0];
} else if (orientation == UIInterfaceOrientationPortraitUpsideDown) {
[filterRef setInputRotation:kGPUImageRotateLeft atIndex:0];
}
}
- (UIInterfaceOrientation)orientationForTrack:(AVAsset *)asset
{
AVAssetTrack *videoTrack = [[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
CGSize size = [videoTrack naturalSize];
CGAffineTransform txf = [videoTrack preferredTransform];
if (size.width == txf.tx && size.height == txf.ty)
return UIInterfaceOrientationLandscapeRight;
else if (txf.tx == 0 && txf.ty == 0)
return UIInterfaceOrientationLandscapeLeft;
else if (txf.tx == 0 && txf.ty == size.width)
return UIInterfaceOrientationPortraitUpsideDown;
else
return UIInterfaceOrientationPortrait;
}
Hope this solves your issue.

add to #Ameet Dhas:
one place to set rotation is where before call next filter.
[currentTarget setInputRotation:outPutRotation atIndex:targetTextureIndex];
[currentTarget newFrameReadyAtTime:currentSampleTime atIndex:targetTextureIndex];

Related

How to detect orientation, if screen is locked in portrait mode?

My application works in both orientation(portrait and landscape) but one of the screen is locked in portrait mode. But I have to set one value in Rotation variable for that particular screen. But I did not find orientation.
So I want to find orientation.
I'm using this below code for lock my screen in portrait mode and it will work.
- (UIInterfaceOrientationMask) supportedInterfaceOrientations {
[super supportedInterfaceOrientations];
return UIInterfaceOrientationMaskPortrait;
}
I'm using this below method to detect orientation but this will not called.
- (void)didRotateFromInterfaceOrientation:(UIInterfaceOrientation)fromInterfaceOrientation
{
UIInterfaceOrientation orientation = [[UIApplication sharedApplication] statusBarOrientation];
if (orientation == UIInterfaceOrientationPortrait || orientation == UIInterfaceOrientationPortraitUpsideDown)
{
NSLog(#"UIInterfaceOrientationPortrait");
}
else
{
NSLog(#"UIInterfaceOrientationland");
}
}
You can detect orientation, if screen is locked in portrait mode. The easy way is "CoreMotion". The code snippet is below.
1-) Firstly, you should add CoreMotion freamework to Build Settings
2-) import CoreMotion freamework
#import <CoreMotion/CoreMotion.h>
3-) added property just like below.
#interface BaseGeneralViewController (){
UIInterfaceOrientation orientationLast, orientationAfterProcess;
CMMotionManager *motionManager;
UIInterfaceOrientation orientationNew;
}
4-) we are created method to initialize
- (void)initializeMotionManager{
motionManager = [[CMMotionManager alloc] init];
motionManager.accelerometerUpdateInterval = .2;
motionManager.gyroUpdateInterval = .2;
[motionManager startAccelerometerUpdatesToQueue:[NSOperationQueue currentQueue]
withHandler:^(CMAccelerometerData *accelerometerData, NSError *error) {
if (!error) {
[self outputAccelerationData:accelerometerData.acceleration];
}
else{
NSLog(#"%#", error);
}
}];
}
5-) declare outputAccelerationData method
- (void)outputAccelerationData:(CMAcceleration)acceleration{
if (acceleration.x >= 0.75 && orientationLast != UIInterfaceOrientationLandscapeLeft) {
orientationNew = UIInterfaceOrientationLandscapeLeft;
[self callQRCodePayment:UIDeviceOrientationLandscapeRight];
}
else if (acceleration.x <= -0.75 && orientationLast != UIInterfaceOrientationLandscapeRight) {
orientationNew = UIInterfaceOrientationLandscapeRight;
[self callQRCodePayment:UIDeviceOrientationLandscapeLeft];
}
else if (acceleration.y <= -0.75) {
//orientationNew = UIInterfaceOrientationPortrait;
//NSLog(#"Portrait");
}
else if (acceleration.y >= 0.75) {
//orientationNew = UIInterfaceOrientationPortraitUpsideDown;
}
else {
// Consider same as last time
return;
}
if (orientationNew == orientationLast)
return;
orientationLast = orientationNew;
}
6-) call initialize method in viewDidload
- (void)viewDidLoad
{
[super viewDidLoad];
[self initializeMotionManager];
}
you can see detail postchange Oriantation
here are methods to detect orientation UIDeviceOrientationIsLandscape and UIDeviceOrientationIsPortrait
if (UIDeviceOrientationIsLandscape([UIDevice currentDevice].orientation))
{
// code here for landscape orientation
}
// And
if (UIDeviceOrientationIsPortrait([UIDevice currentDevice].orientation))
{
// code here for Portrait orientation
}

AVAsset to CIDetectorImageOrientation

I want to run face detection using CoreImage on every frame in an AVAsset and to do that, I need the CIDetectorImageOrientation for every sample buffer in that asset.
The problem is, if I convert a CMSampleBuffer to a UIImage and translate that image to exif orientation using:
var exifOrientation: UInt32 {
switch self.imageOrientation {
case .Up: return 1
case .Down: return 3
case .Left: return 8
case .Right: return 6
case .UpMirrored: return 2
case .DownMirrored: return 4
case .LeftMirrored: return 5
case .RightMirrored: return 7
}
}
the orientation is always .Up, regardless of how the video was taken (front vs back camera) or the orientation the video was taken in.
How can I properly get the CIDetectorImageOrientation for a CMSampleBuffer?
You need to get the orientation of the video track of that asset. This orientation will be the same for every frame.
Here's a roughly how you'd do it in Objective C.
AVAssetTrack *videoTrack =[asset tracksWithMediaType:AVMediaTypeVideo].firstObject;
CGSize size = [videoTrack naturalSize];
CGAffineTransform preferredTransform = [videoTrack preferredTransform];
if (size.width == preferredTransform.tx && size.height == preferredTransform.ty) {
return kCGImagePropertyOrientationRight;
} else if (preferredTransform.tx == 0 && preferredTransform.ty == 0) {
return kCGImagePropertyOrientationUpLeft;
} else if (preferredTransform.tx == 0 && preferredTransform.ty == size.width) {
return kCGImagePropertyOrientationDown;
}
...

Xcode 6 iOS 8 Rotation Issue - Large Blank Space On Screen

I am running into a very strange issue with my iPad app in Xcode 6. Previously (When building with Xcode 5 / iOS 7 SDK) it would handle rotations with no problem, but now that developers are required to build with Xcode 6 / iOS 8 SDK, my app no longer handles rotation properly.
I did some research and was able to use the viewWillTransitionToSize method in order to get all my subviews to properly rotate and resize themselves. However, my screen has a large white rectangle along the side when I rotate. If I start the app in portrait and rotate to landscape, it goes from looking normal to having a large white space on the right.
(The same thing happens if I start in landscape and rotate to portrait, but the white space is on the bottom).
My subviews are definitely resizing themselves properly, but that white space unfortunately covers them up. I checked the both the bounds and frame of my UIScreen window and those values seem valid (1024w x 768h in landscape, 768w x 1024h in portrait). Previously it was a black space but once I added the line [self.window setFrame:[[UIScreen mainScreen] bounds]]; to my didFinishLaunchingWithOptions method, it became a white space.
The following is the viewWillTransitionToSize code that I'm using:
- (void) viewWillTransitionToSize:(CGSize)size withTransitionCoordinator:(id<UIViewControllerTransitionCoordinator>)coordinator
{
[super viewWillTransitionToSize:size WithTransitionCoordinator:coordinator];
UIInterfaceOrientation *orientation = [self interfaceFromTransform:[coordinator targetTransform]];
UIInterfaceOrientation *oldOrientation = [[UIApplication sharedApplication] statusBarOrientation];
[self willRotateToInterfaceOrientation:orientation duration:1.0];
[coordinator animateAlongsideTransition:^(id<UIViewControllerTransitionCoordinatorContext> content)
{
[self willAnimateRotationToInterfaceOrientation:orientation duration:1.0];
}
completion:^(id<UIViewControllerTransitionCoordinatorContext> context)
{
[self didRotateFromInterfaceOrientation:oldOrientation];
}];
CGFloat height = self.view.frame.size.height;
CGFloat width = self.view.frame.size.width;
[self.view setFrame:CGRectMake(0, 0, height, width)];
}
interfaceFromTransform is a method I created to determine which interface the transform is rotating to:
- (UIInterfaceOrientation) interfaceFromTransform: (CGAffineTransform)transform
{
UIInterfaceOrientation old = [[UIApplication sharedApplication] statusBarOrientation];
int rotation = 0;
if (transform.b == -1 && transform.c == 1)
rotation = 90;
if (transform.b == 1 && transform.c == -1)
rotation = -90;
if (transform.a == -1 && transform.d == -1)
rotation = 180;
if (old == UIInterfaceOrientationLandscapeRight)
{
if (rotation == 90)
return UIInterfaceOrientationPortrait;
if (rotation == -90)
return UIInterfaceOrientationPortraitUpsideDown;
if (rotation == 180)
return UIInterfaceOrientationLandscapeLeft;
}
if (old == UIInterfaceOrientationLandscapeLeft)
{
if (rotation == 90)
return UIInterfaceOrientationPortraitUpsideDown;
if (rotation == -90)
return UIInterfaceOrientationPortrait;
if (rotation == 180)
return UIInterfaceOrientationLandscapeRight;
}
if (old == UIInterfaceOrientationPortraitUpsideDown)
{
if (rotation == 90)
return UIInterfaceOrientationLandscapeRight;
if (rotation == -90)
return UIInterfaceOrientationLandscapeLeft;
if (rotation == 180)
return UIInterfaceOrientationPortrait;
}
if (old == UIInterfaceOrientationPortrait)
{
if (rotation == 90)
return UIInterfaceOrientationLandscapeLeft;
if (rotation == -90)
return UIInterfaceOrientationLandscapeRight;
if (rotation == 180)
return UIInterfaceOrientationPortraitUpsideDown;
}
return old;
}
Yet despite all this, that white space still refuses to go away. Is there something obvious that I'm missing that will keep it from appearing when I rotate the screen?
As it turns out, I had disabled "Autoresize Subviews" in my main window nib file. Enabling this immediately fixed the problem.

change screen rotation of app extension in objective c

How do I lock from rotating screen, I want my app extension to be only in portrait mode.
When I'm using my extension inside Photos my I can rotate the screen to landscape.
thanks
You have to choice:
1- Set landscape and portrait as supported interface orientation in the project, and then for each ViewController, you will override the supported interface orientation;
2- Set only portrait mode in the project, and in the ViewController you need it, you can make a 90 degree rotation
self.view.transform = CGAffineTransformMakeRotation(M_PI_2);
EDIT: so you can do this. Set your controller as observer for the keyPath UIDeviceOrientationDidChangeNotification. Then:
-(void)changedOrientation: (NSNotification *)note
{
if (note)
{
UIDevice *dev = (UIDevice *)note.object;
if ([dev orientation] == UIInterfaceOrientationLandscapeRight)
{
self.view.transform = CGAffineTransformMakeRotation(M_PI_2);
}
else if ([dev orientation] == UIInterfaceOrientationPortrait)
{
self.view.transform = CGAffineTransformIdentity;
}
else if ([dev orientation] == UIInterfaceOrientationPortraitUpsideDown)
{
self.view.transform = CGAffineTransformIdentity;
}
else if ([dev orientation] == UIInterfaceOrientationLandscapeLeft)
{
self.view.transform = CGAffineTransformMakeRotation(-M_PI_2);
}
}
}

How to detect if a video file was recorded in portrait orientation, or landscape in iOS

I am using AlAssetsGroup enumerateAssetsAtIndexes to list the assets in the Photos (Camera) app. For a given video asset I want to determine whether it was shot in portrait or landscape mode.
In the following code, asset is an AlAsset and I have tested to see if it is a video asset [asset valueForProperty:ALAssetPropertyType] is AlAssetTypeVideo, then:
int orientation = [[asset valueForProperty:ALAssetPropertyOrientation] intValue];
In this case orientation is always 0 which is ALAssetOrientationUp. Maybe this is to be expected, all videos are up right, but a portrait video is represented in MPEG-4 as a landscape video turned 90 degrees (i.e. all videos are actually landscape, try the MediaInfo app on the mac if you don't believe me).
Where within the file and/or how do I access the information that tells me it was actually recorded while holding the phone in portrait orientation?
I have also tried this, given the url of the asset:
AVURLAsset *avAsset = [[AVURLAsset alloc] initWithURL:url options:nil];
CGSize size = [avAsset naturalSize];
NSLog(#"size.width = %f size.height = %f", size.width, size.height);
CGAffineTransform txf = [avAsset preferredTransform];
NSLog(#"txf.a = %f txf.b = %f txf.c = %f txf.d = %f txf.tx = %f txf.ty = %f",
txf.a, txf.b, txf.c, txf.d, txf.tx, txf.ty);
Which always yields a width > height so for iPhone 4, width=1280 height=720 and the transform a and d values are 1.0, the others are 0.0, regardless of the capture orientation.
I have looked at the meta data using MediaInfo app on the Mac, I have done a Hexdump and so far have not found any difference between a landscape and portrait video. But QuickTime knows and displays portrait videos vertically, and the phone knows by rotating a portrait video if you are holding the phone in landscape orientation on playback and correctly displaying it if holding it in portrait.
BTW I can't use ffmpeg (can't live with the license restrictions). Is there an iPhone SDK native way to do this?
Based on the previous answer, you can use the following to determine the video orientation:
+ (UIInterfaceOrientation)orientationForTrack:(AVAsset *)asset
{
AVAssetTrack *videoTrack = [[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
CGSize size = [videoTrack naturalSize];
CGAffineTransform txf = [videoTrack preferredTransform];
if (size.width == txf.tx && size.height == txf.ty)
return UIInterfaceOrientationLandscapeRight;
else if (txf.tx == 0 && txf.ty == 0)
return UIInterfaceOrientationLandscapeLeft;
else if (txf.tx == 0 && txf.ty == size.width)
return UIInterfaceOrientationPortraitUpsideDown;
else
return UIInterfaceOrientationPortrait;
}
Somebody on apple dev forums suggested getting the transform of the video track, this does the job. You can see from the logs below that for these orientations the results make sense and our web developer is now able to rotate a variety of vids so they all match and composite them into one video.
AVAssetTrack* videoTrack = [[avAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
CGSize size = [videoTrack naturalSize];
NSLog(#"size.width = %f size.height = %f", size.width, size.height);
CGAffineTransform txf = [videoTrack preferredTransform];
NSLog(#"txf.a = %f txf.b = %f txf.c = %f txf.d = %f txf.tx = %f txf.ty = %f", txf.a, txf.b, txf.c, txf.d, txf.tx, txf.ty);
Logs using 4 iPhone 4 videos with the normal cam:
(1) landscape cam on right side (home button on left)
(2) landscape left
(3) portrait upside-down
(4) portrait up-right (home button at bottom)
2011-01-07 20:07:30.024 MySecretApp[1442:307] size.width =
1280.000000 size.height = 720.000000 2011-01-07 20:07:30.027
MySecretApp[1442:307] txf.a =
-1.000000 txf.b = 0.000000 txf.c = 0.000000 txf.d = -1.000000 txf.tx = 1280.000000 txf.ty = 720.000000
2011-01-07 20:07:45.052 MySecretApp[1442:307] size.width =
1280.000000 size.height = 720.000000 2011-01-07 20:07:45.056
MySecretApp[1442:307] txf.a = 1.000000
txf.b = 0.000000 txf.c = 0.000000
txf.d = 1.000000 txf.tx = 0.000000
txf.ty = 0.000000
2011-01-07 20:07:53.763 MySecretApp[1442:307] size.width =
1280.000000 size.height = 720.000000 2011-01-07 20:07:53.766
MySecretApp[1442:307] txf.a = 0.000000
txf.b = -1.000000 txf.c = 1.000000
txf.d = 0.000000 txf.tx = 0.000000
txf.ty = 1280.000000
2011-01-07 20:08:03.490 MySecretApp[1442:307] size.width =
1280.000000 size.height = 720.000000 2011-01-07 20:08:03.493
MySecretApp[1442:307] txf.a = 0.000000
txf.b = 1.000000 txf.c = -1.000000
txf.d = 0.000000 txf.tx = 720.000000
txf.ty = 0.000000
In my use case I only needed to know if a video is in portrait or not (landscape).
guard let videoTrack = AVAsset(url: videoURL).tracks(withMediaType: AVMediaTypeVideo).first else {
return ...
}
let transformedVideoSize = videoTrack.naturalSize.applying(videoTrack.preferredTransform)
let videoIsPortrait = abs(transformedVideoSize.width) < abs(transformedVideoSize.height)
This has been tested with both front and rear cameras for all orientation possibilities.
AVAssetImageGenerator
If you are using AVAssetImageGenerator to generate images from AVAssets, you can simply set the .appliesPreferredTrackTransform property of AVAssetImageGenerator to true and it will return you images from your asset in the correct orientation! :)
Swift 3
But to extend on #onmyway133's answer in Swift 3:
import UIKit
import AVFoundation
extension AVAsset {
var g_size: CGSize {
return tracks(withMediaType: AVMediaTypeVideo).first?.naturalSize ?? .zero
}
var g_orientation: UIInterfaceOrientation {
guard let transform = tracks(withMediaType: AVMediaTypeVideo).first?.preferredTransform else {
return .portrait
}
switch (transform.tx, transform.ty) {
case (0, 0):
return .landscapeRight
case (g_size.width, g_size.height):
return .landscapeLeft
case (0, g_size.width):
return .portraitUpsideDown
default:
return .portrait
}
}
}
If you do not want to use AVFoundation Framework just to get the orientation of the recorded video then try it out
-(void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo: (NSDictionary *)info {
NSString *orientation;
NSString *videoPath = [[info objectForKey:UIImagePickerControllerMediaURL] path];
NSURL *myURL = [[NSURL alloc] initFileURLWithPath:videoPath];
self.movieController = [[MPMoviePlayerController alloc] initWithContentURL:myURL];
UIImage *thumbImage = [movieController thumbnailImageAtTime:1.0 timeOption:MPMovieTimeOptionNearestKeyFrame];
float width = thumbImage.size.width;
float height = thumbImage.size.height;
if (width > height){
orientation = #"Landscape";
}else{
orientation = #"Portrait";
}
[self dismissViewControllerAnimated:YES completion:nil];
}
While several of the answers here are correct, they are not comprehensive. For instance, you may also need to know which camera device was used to apply the proper transforms. I created a gist to do this very thing; extract the UIInterfaceOrientation and the AVCaptureDevicePosition.
Extract AVAsset Orientation and Camera Position
- (UIImageOrientation)getImageOrientationWithVideoOrientation:(UIInterfaceOrientation)videoOrientation {
UIImageOrientation imageOrientation;
switch (videoOrientation) {
case UIInterfaceOrientationLandscapeLeft:
imageOrientation = UIImageOrientationUp;
break;
case UIInterfaceOrientationLandscapeRight:
imageOrientation = UIImageOrientationDown;
break;
case UIInterfaceOrientationPortrait:
imageOrientation = UIImageOrientationRight;
break;
case UIInterfaceOrientationPortraitUpsideDown:
imageOrientation = UIImageOrientationLeft;
break;
}
return imageOrientation;
}
Extending on George's answer in Swift 2
UIInterfaceOrientation is the same with UIImageOrientation
extension AVAsset {
var g_size: CGSize {
return tracksWithMediaType(AVMediaTypeVideo).first?.naturalSize ?? .zero
}
var g_orientation: UIInterfaceOrientation {
guard let transform = tracksWithMediaType(AVMediaTypeVideo).first?.preferredTransform else {
return .Portrait
}
switch (transform.tx, transform.ty) {
case (0, 0):
return .LandscapeRight
case (g_size.width, g_size.height):
return .LandscapeLeft
case (0, g_size.width):
return .PortraitUpsideDown
default:
return .Portrait
}
}
}
Try this in swift 5
func checkVideoOrientation(url: URL) {
let asset = AVAsset(url: url)
let track = asset.tracks(withMediaType: .video).first
let size = track?.naturalSize ?? .zero
let transform = track?.preferredTransform ?? .identity
if size.width == transform.tx && size.height == transform.ty {
self.postVideoOrientation = false //Portrait
} else if size.width == transform.ty && size.height == transform.tx {
self.postVideoOrientation = true //"Landscape"
} else {
self.postVideoOrientation = false //"Unknown"
}
}

Resources