AVCaptureVideoPreviewLayer wrong orientation when UI rotation is disabled - ios

I have a AVCaptureVideoPreviewLayer instance added to a view controller view hierarchy.
- (void) loadView {
...
self.previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:nil];
self.previewLayer.frame = self.view.bounds;
self.previewLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
[self.view.layer addSublayer: _previewLayer];
// adding other UI elements
...
}
...
- (void) _setupPreviewLayerWithSession: (AVCaptureSession*) captureSession
{
self.previewLayer.session = self.captureManager.captureSession;
self.previewLayer.connection.videoOrientation = AVCaptureVideoOrientationLandscapeRight;
}
The layer frame is updated in -viewDidLayoutSubviews method. View controller orientation is locked to UIInterfaceOrientationMaskLandscapeRight.
The issue is as following:
the device is held in landscape orientation
the view controller is presented modally - video layer displays correctly.
the device is then locked and while it's locked the device is rotated to portrait orientation.
the device is then unlocked while still being in portrait orientation and for several seconds the video layer is displayed rotated 90 degrees. However, the frame for the video layer is correct. All other UI elements display correctly. After several seconds the layer snaps to correct orientation.
Please find the bounds for the layer and UI elements below
I've tried updating the video layer orientation as following (with no results):
subscribing to AVCaptureSessionDidStartRunningNotification and UIApplicationDidBecomeActiveNotification notifications
calling the update when -viewWillTransitionToSize:withTransitionCoordinator: method is called
on -viewWillAppear:
The issue doesn't seem to be connected to video layer orientation itself, but more to view hierarchy layout.
UPDATE:
As suggested, I also tried updating the video layer orientation on device orientation change which didn't help.
I also noticed that the issue mostly happens after the application is launched and the screen is presented for the first time. On subsequent screen presentations during the same session, the reproduce rate for the issue is really low (something like 1/20).

Try this code:
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(orientationChanged:)
name:UIDeviceOrientationDidChangeNotification
object:nil];
-(void)orientationChanged:(NSNotification *)notif {
[_videoPreviewLayer setFrame:_viewPreview.layer.bounds];
if (_videoPreviewLayer.connection.supportsVideoOrientation) {
_videoPreviewLayer.connection.videoOrientation = [self interfaceOrientationToVideoOrientation:[UIApplication sharedApplication].statusBarOrientation];
}
}
- (AVCaptureVideoOrientation)interfaceOrientationToVideoOrientation:(UIInterfaceOrientation)orientation {
switch (orientation) {
case UIInterfaceOrientationPortrait:
return AVCaptureVideoOrientationPortrait;
case UIInterfaceOrientationPortraitUpsideDown:
return AVCaptureVideoOrientationPortraitUpsideDown;
case UIInterfaceOrientationLandscapeLeft:
return AVCaptureVideoOrientationLandscapeLeft;
case UIInterfaceOrientationLandscapeRight:
return AVCaptureVideoOrientationLandscapeRight;
default:
break;
}
// NSLog(#"Warning - Didn't recognise interface orientation (%d)",orientation);
return AVCaptureVideoOrientationPortrait;
}

Related

How to make `AVCaptureVideoPreviewLayer` clip to the view border during rotation

I add a AVCaptureVideoPreviewLayer pretty much based on Apple's documentation about AVCaptureVideoPreviewLayer as the following:
AVCaptureSession *captureSession = <#Get a capture session#>;<br>
AVCaptureVideoPreviewLayer *previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:captureSession];<br>
UIView *aView = <#The view in which to present the layer#>;<br>
previewLayer.frame = aView.bounds; // Assume you want the preview layer to fill the view.<br>
[aView.layer addSublayer:previewLayer];
I added the didRotateFromInterfaceOrientation function in order to handle the rotation with the following code:
- (void)didRotateFromInterfaceOrientation:(UIInterfaceOrientation)fromInterfaceOrientation
{
[super didRotateFromInterfaceOrientation:fromInterfaceOrientation];
[self.videoPreviewLayer setFrame:self.view.layer.bounds];
[self.videoPreviewLayer.connection setVideoOrientation:[self getAVCaptureVideoOrientation]];
}
- (AVCaptureVideoOrientation)getAVCaptureVideoOrientation {
UIDeviceOrientation deviceOrientation = [[UIDevice currentDevice] orientation];
if ( deviceOrientation == UIDeviceOrientationLandscapeLeft )
return AVCaptureVideoOrientationLandscapeRight;
else if
// and other 3 orientations
// ...
}
The result looks okay but but animation not so good. The camera layer doesn't start resizing until the rotation finished and I can see the background view during the rotation.
http://yzhong.co/wp-content/uploads/2015/03/My_Movie_1_-_Small.gif
Then I start looking around and found these two SO questions: Problems automatically resizing AVCaptureVideoPreviewLayer on rotation and AVCaptureVideoPreviewLayer smooth orientation rotation. I tried the solutions proposed in those question and added the following function:
- (void)willAnimateRotationToInterfaceOrientation:(UIInterfaceOrientation)toInterfaceOrientation duration:(NSTimeInterval)duration {
if (_videoPreviewLayer) {
if (toInterfaceOrientation==UIInterfaceOrientationPortrait) {
[self.videoPreviewLayer setAffineTransform:CGAffineTransformMakeRotation(0)];
} else if (toInterfaceOrientation==UIInterfaceOrientationLandscapeLeft) {
[self.videoPreviewLayer setAffineTransform:CGAffineTransformMakeRotation(M_PI/2)];
} else if (toInterfaceOrientation==UIInterfaceOrientationLandscapeRight) {
[self.videoPreviewLayer setAffineTransform:CGAffineTransformMakeRotation(-M_PI/2)];
} else {
[self.videoPreviewLayer setAffineTransform:CGAffineTransformMakeRotation(M_PI)];
}
self.videoPreviewLayer.frame = self.view.bounds;
}
}
And I get this following, which is a lot better. However, the animation is still a little weird. It seems like it is rotating backward and then forward to the right position? Now I have no clue what to do next.
http://yzhong.co/wp-content/uploads/2015/03/My_Movie_2_-_Small.gif
What I am looking for is really very simple. I want to have a nice, smooth and natural transition/rotation such as the Apple's default camera app does. Could anyone tell me what I am doing wrong here?
http://yzhong.co/wp-content/uploads/2015/03/My_Movie_3_-_Small.gif
Please help!

avcapture video repeated part on changing orientation

I have got an issue regarding video capture orientation. first of all its about voip app which uses pjsip which can transmit video. the video is being captured using AVCapture frame. so the issue arises while device orientation changes then i have to set the avcapture orientation as well.
for example:
capConnection.videoOrientation = AVCaptureVideoOrientationLandscapeLeft;
it works fine but i have repeat image part.
so the question is how to get rid of this repeated image part. i have tried this solution but it keeps crashing on vImageRotate90_ARGB8888 any idea how resolve this issue?
in order to try this yourself out, you can get PJSIP version 2.3 video with sample project compile and run it against a test SIP server.
edit: the preview layer is rotated and scaled fine. the particular issue happens on receiving RTP (video) stream when that device rotates and sends images with repeated edges. for instance, if iPadA(horizontal) starts video call with iPadB(horizontal) the image is fine and not repeated edges. but if the iPadA rotates to vertical then iPadB gets this repeated edge images. notice on rotation the capture connection orientation is set to current device orientation.
note the the preview layer has AVLayerVideoGravityResize but that does not affect the outgoing video stream.
The two key pieces are setting the autoresizingMask in viewDidLoad and adjusting your captureVideoPreviewLayer.frame to self.view.layer.bounds; in willAnimateRotationToInterfaceOrientation.
- (void)viewDidLoad
{
[super viewDidLoad];
self.view.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
//
// react to device orientation notifications
//
[[NSNotificationCenter defaultCenter] addObserver : self
selector : #selector(deviceOrientationDidChange:)
name : UIDeviceOrientationDidChangeNotification
object : nil];
[[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
}
- (void) willAnimateRotationToInterfaceOrientation : (UIInterfaceOrientation)toInterfaceOrientation
duration : (NSTimeInterval)duration
{
captureVideoPreviewLayer.frame = self.view.layer.bounds;
[[captureVideoPreviewLayer connection] setVideoOrientation:toInterfaceOrientation];
}
- (void)deviceOrientationDidChange: (NSNotification*)notification
{
UIDeviceOrientation orientation = [UIDevice currentDevice].orientation;
switch (orientation)
{
case UIDeviceOrientationPortrait:
case UIDeviceOrientationPortraitUpsideDown:
case UIDeviceOrientationLandscapeLeft:
case UIDeviceOrientationLandscapeRight:
currentDeviceOrientation = orientation;
break;
// unsupported?
case UIDeviceOrientationFaceUp:
case UIDeviceOrientationFaceDown:
default:
break;
}
}

AVCaptureVideoPreviewLayer, video orientation stuck

I have an iPhone camera app in which I use the AVCaptureVideoPreviewLayer. When someone takes a picture/stillImage this is shown in a new ViewController. The magic happens once this view controller is dismissed.
The whole app rotates properly, both viewcontroller; but for one exception. If I take a picture in one orientation, rotate the device in the new ViewController and then go back to the AVCaptureVideoPreviewLayer the whole layer rotates along just fine, except for the image, so suddenly the input is presented sideways through the previewLayer.
I have checked, and tried setting the frame for the PreviewLayer, but this all seems fine, the values I see when debugging are all correct. It is just the image that is displayed that is skewed. Rotating the device back and forth fixes this issue during use.
Has anyone seen this before, and does anyone have a clue how to fix this?
You could change the previewLayer's connection's videoOrientation when the interface rotates
- (void)didRotateFromInterfaceOrientation:(UIInterfaceOrientation)fromInterfaceOrientation {
self.previewLayer.connection.videoOrientation = self.interfaceOrientation;
}
Since didRotateFromInterfaceOrientation: is deprecated in iOS 8, I change the video orientation of the connection in viewDidLayoutSubviews
switch ([UIApplication sharedApplication].statusBarOrientation) {
case UIInterfaceOrientationPortrait:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationPortrait;
break;
case UIInterfaceOrientationPortraitUpsideDown:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationPortraitUpsideDown;
break;
case UIInterfaceOrientationLandscapeLeft:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationLandscapeLeft;
break;
case UIInterfaceOrientationLandscapeRight:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationLandscapeRight;
break;
default:
break;
}
I had a similar problem in the past.
Once dismissed camera controller you could refresh view rotation with this:
UIDeviceOrientation orientation = [[UIDevice currentDevice] orientation];
UIWindow *window = [[UIApplication sharedApplication] keyWindow];
UIView *topView = [window.subviews objectAtIndex:0];
[topView removeFromSuperview];
[window addSubview:topView];
So, removing top most view on shown window and resetting it on window, it should refresh view rotation properly.
Hope it helps
Maybe, instead of changing AVCaptureVideoPreviewLayer frame, you could try to put it in a container UIView, and set this container's frame. (I fixed a 'sizing' bug on ÀVCaptureVideoPreviewLayerthis way : sizing directly PreviewLayer had no effect, but sizing its containingUIView` worked).
Or,
maybe you can force orientation in your viewController viewWillAppear (or viewDidAppear ?)
- (void)viewWillAppear:(BOOL)animated
{
[super viewWillAppear:animated];
// try and force current orientation
UIApplication *application = [UIApplication sharedApplication];
UIInterfaceOrientation currentOrientation = application.statusBarOrientation;
[application setStatusBarOrientation:currentOrientation animated:animated];
}
Good luck !
Since the interfaceOrientation is deprecated, the up to date swift 2.0 solution shall be:
let previewLayerConnection = self.previewLayer.connection
let orientation = AVCaptureVideoOrientation(rawValue: UIApplication.sharedApplication().statusBarOrientation.rawValue)
previewLayerConnection.videoOrientation = orientation!

When does it changes orientation to default orientation?

I'm developing an iOS application with latest SDK.
My app only supports Landscape right orientation and it only has one view controller. I set the only available orientation modifying myProject-info.plist.
I'm trying to understand why this code outputs a log for a view in Portrait mode when I only supports landscape right orientation.
- (void) viewWillAppear:(BOOL)animated
{
[super viewWillAppear:animated];
UIInterfaceOrientation orientation = [UIApplication sharedApplication].statusBarOrientation;
UIView *videoView = self.videoPreviewView;
NSLog(#"Video frame: %#", NSStringFromCGRect(videoView.frame));
}
Console output:
Video frame: {{0, 0}, {320, 568}}
I'm working with AVCaptureVideoPreviewLayer and I need to set its frame, but when I do it on viewWillAppear: I get portrait values but orientation value is Landscape right.
Is there an event or anything similar that i gets triggered when orientation changes to landscape?
When does the app changes orientation to default orientation?
Go into xCode, and click the project name at the top of the screen (with an xcode LOGO underneath it).
Then reveal the iPhone/Ipad development info and select the orientations you want to be available for users :)
If you add the line:
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(orientationChanged) name:UIDeviceOrientationDidChangeNotification object:nil];
then it will call the -(void)orientationChanged; method every time the orientation is changed. You can then do something like:
-(void)orientationChanged {
if (UIDeviceOrientationIsPortrait([UIApplication sharedApplication].statusBarOrientation) {
//device is portrait;
}
else {
//device is landscape;
}
}
Then you can use this to determine the layout of your views and setup based on the orientation.
EDIT:
You can also check if the orientation returned by the statusBar's orientation is equal to UIDeviceOrientationLandscapeLeft/LandscapeRight/Portrait/PortraitUpsideDown.

Problems automatically resizing AVCaptureVideoPreviewLayer on rotation

I've programmed a static framework, and therein I have a class called ViewfinderViewController, that set's up the camera with an AVCaptureSession. This ViewController also add's an AVCaptureVideoPreviewLayer as sublayer to its own view's layer:
// code in ViewfinderViewController.m
previewLayer = [[AVCaptureVideoPreviewLayer alloc] initWithSession:captureSession];
previewLayer.frame = self.view.bounds;
previewLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
[self.view.layer addSublayer:previewLayer];
The ViewfinderViewController also does some video processing, and offers user's of the framework to register for some delegate methods, in case something of interest was found or was not found in one of the processed frames.
In the viewDidLoad method of my real application, I create an instance of ViewfinderViewController, set it's frame and add it's view as a subview:
// code in the app's view controller viewDidLoad method
ViewfinderViewController *vfVC = [[ViewfinderViewController alloc] init];
vfVC.view.frame = self.view.bounds;
[self.view addSubview:vfVC.view];
This works fine, as long as I disregard rotation (which I did until now). My main problem is, that ViewfinderViewController.view resizes its frame, but the AVCaptureVideoPreviewLayer does not. I'm a bit puzzeled: Shouldn't the previewLayer also resize automatically, as the view corresponding to it's superLayer is resized correctly?
Any ideas? I'll gladly provide more information, but to keep the question shorter I won't go any further for now. Thanks.
Finally, I've found the following to be working best for my app.
In my application, I add the ViewFinderViewController now as child view controller:
- (void)viewWillAppear:(BOOL)animated;
{
if ([self.childViewControllers containsObject:self.viewfinderViewController] == NO) {
[self addChildViewController:self.viewfinderViewController];
self.viewfinderViewController.view.frame = self.view.bounds;
[self.view addSubview:self.viewfinderViewController.view];
[self.viewfinderViewController didMoveToParentViewController:self];
}
}
This way, the willAnimateRotationToInterfaceOrientation: method will be called automatically, which I previously forwarded from the view controller that added the viewfinder's view as it's subview.
Furthermore, I update the previewLayer size in this method of ViewfinderViewController:
- (void)viewWillLayoutSubviews;
{
self.previewLayer.frame = self.view.bounds;
}
I handle rotation of the previewLayer by implementing these two methods:
- (void)willAnimateRotationToInterfaceOrientation:(UIInterfaceOrientation)toInterfaceOrientation duration:(NSTimeInterval)duration {
[CATransaction begin];
[CATransaction setAnimationDuration:duration];
[CATransaction setAnimationTimingFunction:[CAMediaTimingFunction functionWithName:kCAMediaTimingFunctionEaseInEaseOut]];
[self updatePreviewLayerForOrientation:toInterfaceOrientation];
[CATransaction commit];
}
- (void)updatePreviewLayerForOrientation:(UIInterfaceOrientation)interfaceOrientation;
{
// correct position of previewLayer
CGSize newSize = self.view.bounds.size;
self.previewLayer.position = CGPointMake(0.5 * newSize.width, 0.5 * newSize.height);
// rotate the previewLayer, in order to have camera picture right
switch (interfaceOrientation) {
case UIInterfaceOrientationPortrait:
[self.previewLayer setAffineTransform:CGAffineTransformMakeRotation(0)];
break;
case UIInterfaceOrientationLandscapeLeft:
[self.previewLayer setAffineTransform:CGAffineTransformMakeRotation(M_PI/2)];
break;
case UIInterfaceOrientationLandscapeRight:
[self.previewLayer setAffineTransform:CGAffineTransformMakeRotation(-M_PI/2)];
break;
case UIDeviceOrientationPortraitUpsideDown:
[self.previewLayer setAffineTransform:CGAffineTransformMakeRotation(M_PI)];
break;
default:
break;
}
}
I hope this helps everybody who is adding AVCaptureVideoPreviewLayer as a sublayer and has problems with resizing and orientation when it comes to interface rotation.

Resources