Upon looking into RTCCameraVideoCapturer.m I found this:
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection {
#if TARGET_OS_IPHONE
switch (_orientation) {
case UIDeviceOrientationPortrait:
_rotation = RTCVideoRotation_90;
break;
case UIDeviceOrientationPortraitUpsideDown:
_rotation = RTCVideoRotation_270;
break;
case UIDeviceOrientationLandscapeLeft:
_rotation = usingFrontCamera ? RTCVideoRotation_180 : RTCVideoRotation_0;
break;
case UIDeviceOrientationLandscapeRight:
_rotation = usingFrontCamera ? RTCVideoRotation_0 : RTCVideoRotation_180;
break;
}
#else
// No rotation on Mac.
_rotation = RTCVideoRotation_0;
#endif
}
Why does a default camera capturer's orientation is 90°. I made a simple voip client and the videofrom my iPhone looks rotated even when the device is in Portrait. What am I missing here? Is there any way to rotate capturer's video orientation.
Any thoughts appreciated
Check AVCaptureConnection.videoOrientation.
https://developer.apple.com/documentation/avfoundation/avcaptureconnection/1389415-videoorientation
I guess the raw orientation depends of the camera manufacturer. To offer maximum speed to client, image data are transferred straight from sensor buffer. You can't rotate capturer's video orientation. You can only mix video and interface orientation properties to properly present it to the user.
Related
I have a AVCaptureVideoPreviewLayer instance added to a view controller view hierarchy.
- (void) loadView {
...
self.previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:nil];
self.previewLayer.frame = self.view.bounds;
self.previewLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
[self.view.layer addSublayer: _previewLayer];
// adding other UI elements
...
}
...
- (void) _setupPreviewLayerWithSession: (AVCaptureSession*) captureSession
{
self.previewLayer.session = self.captureManager.captureSession;
self.previewLayer.connection.videoOrientation = AVCaptureVideoOrientationLandscapeRight;
}
The layer frame is updated in -viewDidLayoutSubviews method. View controller orientation is locked to UIInterfaceOrientationMaskLandscapeRight.
The issue is as following:
the device is held in landscape orientation
the view controller is presented modally - video layer displays correctly.
the device is then locked and while it's locked the device is rotated to portrait orientation.
the device is then unlocked while still being in portrait orientation and for several seconds the video layer is displayed rotated 90 degrees. However, the frame for the video layer is correct. All other UI elements display correctly. After several seconds the layer snaps to correct orientation.
Please find the bounds for the layer and UI elements below
I've tried updating the video layer orientation as following (with no results):
subscribing to AVCaptureSessionDidStartRunningNotification and UIApplicationDidBecomeActiveNotification notifications
calling the update when -viewWillTransitionToSize:withTransitionCoordinator: method is called
on -viewWillAppear:
The issue doesn't seem to be connected to video layer orientation itself, but more to view hierarchy layout.
UPDATE:
As suggested, I also tried updating the video layer orientation on device orientation change which didn't help.
I also noticed that the issue mostly happens after the application is launched and the screen is presented for the first time. On subsequent screen presentations during the same session, the reproduce rate for the issue is really low (something like 1/20).
Try this code:
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(orientationChanged:)
name:UIDeviceOrientationDidChangeNotification
object:nil];
-(void)orientationChanged:(NSNotification *)notif {
[_videoPreviewLayer setFrame:_viewPreview.layer.bounds];
if (_videoPreviewLayer.connection.supportsVideoOrientation) {
_videoPreviewLayer.connection.videoOrientation = [self interfaceOrientationToVideoOrientation:[UIApplication sharedApplication].statusBarOrientation];
}
}
- (AVCaptureVideoOrientation)interfaceOrientationToVideoOrientation:(UIInterfaceOrientation)orientation {
switch (orientation) {
case UIInterfaceOrientationPortrait:
return AVCaptureVideoOrientationPortrait;
case UIInterfaceOrientationPortraitUpsideDown:
return AVCaptureVideoOrientationPortraitUpsideDown;
case UIInterfaceOrientationLandscapeLeft:
return AVCaptureVideoOrientationLandscapeLeft;
case UIInterfaceOrientationLandscapeRight:
return AVCaptureVideoOrientationLandscapeRight;
default:
break;
}
// NSLog(#"Warning - Didn't recognise interface orientation (%d)",orientation);
return AVCaptureVideoOrientationPortrait;
}
I am trying to make a video rotate and scale larger when the user rotates the screen to landscape.
- (void) orientationChanged:(NSNotification *)note
{
bool switchedLeft;
UIDevice * device = note.object;
switch(device.orientation)
{
case UIDeviceOrientationPortrait:
self.videoView.transform=CGAffineTransformMakeScale(0.5,0.5);
if (switchedLeft) {
self.videoView.transform=CGAffineTransformMakeRotation(-M_PI_2);
}else{
self.videoView.transform=CGAffineTransformMakeRotation(M_PI_2);
}
break;
case UIDeviceOrientationLandscapeLeft:
self.videoView.transform=CGAffineTransformMakeRotation(M_PI_2);
self.videoView.transform=CGAffineTransformMakeScale(2.0, 2.0);
switchedLeft=true;
break;
case UIDeviceOrientationLandscapeRight:
self.videoView.transform=CGAffineTransformMakeRotation(-M_PI_2);
self.videoView.transform=CGAffineTransformMakeScale(2.0, 2.0);
switchedLeft=false;
break;
default:
break;
};
}
There are a number of problems. First when I initially rotate to landscape it only does one transformation, in this configuration it just scales it.
The second problem is when I rotate to portrait it calls for the rotation but it never rotates. However i can go back and forth between landscape left and landscape right and it rotates properly. Any help would be greatly appreciated
You are essentially replacing the rotation transform with scale transform. In order to apply both, you need to use CGAffineTransformConcat().
CGAffineTransform rotate = CGAffineTransformMakeRotation(M_PI_2);
CGAffineTransform scale = CGAffineTransformMakeScale(2.0, 2.0);
self.videoView.transform = CGAffineTransformConcat(rotate, scale);
As for the second part, you don't need to apply another rotation, instead set it to default using CGAffineTransformIdentity.
case UIDeviceOrientationPortrait:
CGAffineTransform scale = CGAffineTransformMakeScale(0.5,0.5);
self.videoView.transform = CGAffineTransformConcat(CGAffineTransformIdentity, scale);
break;
try this
CGAffineTransform transform = CGAffineTransformRotate(self. videoView.transform, M_PI);
self. videoView.transform = transform;
I have got an issue regarding video capture orientation. first of all its about voip app which uses pjsip which can transmit video. the video is being captured using AVCapture frame. so the issue arises while device orientation changes then i have to set the avcapture orientation as well.
for example:
capConnection.videoOrientation = AVCaptureVideoOrientationLandscapeLeft;
it works fine but i have repeat image part.
so the question is how to get rid of this repeated image part. i have tried this solution but it keeps crashing on vImageRotate90_ARGB8888 any idea how resolve this issue?
in order to try this yourself out, you can get PJSIP version 2.3 video with sample project compile and run it against a test SIP server.
edit: the preview layer is rotated and scaled fine. the particular issue happens on receiving RTP (video) stream when that device rotates and sends images with repeated edges. for instance, if iPadA(horizontal) starts video call with iPadB(horizontal) the image is fine and not repeated edges. but if the iPadA rotates to vertical then iPadB gets this repeated edge images. notice on rotation the capture connection orientation is set to current device orientation.
note the the preview layer has AVLayerVideoGravityResize but that does not affect the outgoing video stream.
The two key pieces are setting the autoresizingMask in viewDidLoad and adjusting your captureVideoPreviewLayer.frame to self.view.layer.bounds; in willAnimateRotationToInterfaceOrientation.
- (void)viewDidLoad
{
[super viewDidLoad];
self.view.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
//
// react to device orientation notifications
//
[[NSNotificationCenter defaultCenter] addObserver : self
selector : #selector(deviceOrientationDidChange:)
name : UIDeviceOrientationDidChangeNotification
object : nil];
[[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
}
- (void) willAnimateRotationToInterfaceOrientation : (UIInterfaceOrientation)toInterfaceOrientation
duration : (NSTimeInterval)duration
{
captureVideoPreviewLayer.frame = self.view.layer.bounds;
[[captureVideoPreviewLayer connection] setVideoOrientation:toInterfaceOrientation];
}
- (void)deviceOrientationDidChange: (NSNotification*)notification
{
UIDeviceOrientation orientation = [UIDevice currentDevice].orientation;
switch (orientation)
{
case UIDeviceOrientationPortrait:
case UIDeviceOrientationPortraitUpsideDown:
case UIDeviceOrientationLandscapeLeft:
case UIDeviceOrientationLandscapeRight:
currentDeviceOrientation = orientation;
break;
// unsupported?
case UIDeviceOrientationFaceUp:
case UIDeviceOrientationFaceDown:
default:
break;
}
}
The issue that I'm running into is that when a user takes a photo with our app, using AVCaptureSession, I have no way of determining whether they took the photo in Portrait or Landscape mode. Our app only supports Portrait and I keep the Orientation Lock on when using my phone so I'm trying to build a solution assuming that others might do the same.
I looked into using [UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications] but when the Orientation Lock is on, no notifications are ever received. I know that this functionality is possible because the base Camera app and the camera in the Google Hangouts app can detect the rotation (animations on the Cancel and Flash buttons are apparent) when my phone has Orientation Lock on.
Is my best bet to use the accelerometer and detect the angle the phone is being rotated to? An old answer, Detect iPhone screen orientation, makes it very obvious that detecting the angle that way is easy to to do (obviously adapting the answer to use Core Motion instead of UIAccelerometer), but I'm curious if there is another way to do it.
Yes you can do it by looking at the metadata for the image. Don't have time to write up a detailed answer (sorry about that), but I did it for my own project a while back through CMCopyDictionaryOfAttachments(NULL, buffer, kCMAttachmentMode_ShouldPropagate); where I passed in a CMSampleBufferRef for the buffer. I got that buffer from
captureStillImageAsynchronouslyFromConnection:stillImageConnection
completionHandler: ^(CMSampleBufferRef imageDataSampleBuffer, NSError *error){},
but you can get it from
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection as well.
You can find all the keys for that dictionary here.
Did a quick test with the default camera app with the orientation lock on, and I did get a different orientation for the two pictures. 6 for the portrait one, and 3 for the landscape one.
Again, would love to give you more details about this, but I'm sure you can figure it out by looking through the docs.
You can use CoreMotion to detect the device orientation when users locks the orientation. Below is the code:
import CoreMotion
....
func initializeMotionManager() {
motionManager = CMMotionManager()
motionManager?.accelerometerUpdateInterval = 0.2
motionManager?.gyroUpdateInterval = 0.2
}
func startMotionManagerAccelertion() {
motionManager?.startAccelerometerUpdates(to: (OperationQueue.current)!, withHandler: {
(accelerometerData, error) -> Void in
if error == nil {
self.outputAccelertionData((accelerometerData?.acceleration)!)
}
else {
print("\(error!)")
}
})
}
func stopMotionManagerAccelertion(){
motionManager?.stopAccelerometerUpdates()
}
func outputAccelertionData(_ acceleration: CMAcceleration) {
var orientationNew: AVCaptureVideoOrientation
if acceleration.x >= 0.75 {
orientationNew = .landscapeLeft
print("landscapeLeft")
} else if acceleration.x <= -0.75 {
orientationNew = .landscapeRight
print("landscapeRight")
} else if acceleration.y <= -0.75 {
orientationNew = .portrait
print("portrait")
} else if acceleration.y >= 0.75 {
orientationNew = .portraitUpsideDown
print("portraitUpsideDown")
} else {
// Consider same as last time
return
}
if orientationNew == orientationLast { return }
orientationLast = orientationNew
}
I have an iPhone camera app in which I use the AVCaptureVideoPreviewLayer. When someone takes a picture/stillImage this is shown in a new ViewController. The magic happens once this view controller is dismissed.
The whole app rotates properly, both viewcontroller; but for one exception. If I take a picture in one orientation, rotate the device in the new ViewController and then go back to the AVCaptureVideoPreviewLayer the whole layer rotates along just fine, except for the image, so suddenly the input is presented sideways through the previewLayer.
I have checked, and tried setting the frame for the PreviewLayer, but this all seems fine, the values I see when debugging are all correct. It is just the image that is displayed that is skewed. Rotating the device back and forth fixes this issue during use.
Has anyone seen this before, and does anyone have a clue how to fix this?
You could change the previewLayer's connection's videoOrientation when the interface rotates
- (void)didRotateFromInterfaceOrientation:(UIInterfaceOrientation)fromInterfaceOrientation {
self.previewLayer.connection.videoOrientation = self.interfaceOrientation;
}
Since didRotateFromInterfaceOrientation: is deprecated in iOS 8, I change the video orientation of the connection in viewDidLayoutSubviews
switch ([UIApplication sharedApplication].statusBarOrientation) {
case UIInterfaceOrientationPortrait:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationPortrait;
break;
case UIInterfaceOrientationPortraitUpsideDown:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationPortraitUpsideDown;
break;
case UIInterfaceOrientationLandscapeLeft:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationLandscapeLeft;
break;
case UIInterfaceOrientationLandscapeRight:
self.captureVideoPreviewLayer.connection.videoOrientation = AVCaptureVideoOrientationLandscapeRight;
break;
default:
break;
}
I had a similar problem in the past.
Once dismissed camera controller you could refresh view rotation with this:
UIDeviceOrientation orientation = [[UIDevice currentDevice] orientation];
UIWindow *window = [[UIApplication sharedApplication] keyWindow];
UIView *topView = [window.subviews objectAtIndex:0];
[topView removeFromSuperview];
[window addSubview:topView];
So, removing top most view on shown window and resetting it on window, it should refresh view rotation properly.
Hope it helps
Maybe, instead of changing AVCaptureVideoPreviewLayer frame, you could try to put it in a container UIView, and set this container's frame. (I fixed a 'sizing' bug on ÀVCaptureVideoPreviewLayerthis way : sizing directly PreviewLayer had no effect, but sizing its containingUIView` worked).
Or,
maybe you can force orientation in your viewController viewWillAppear (or viewDidAppear ?)
- (void)viewWillAppear:(BOOL)animated
{
[super viewWillAppear:animated];
// try and force current orientation
UIApplication *application = [UIApplication sharedApplication];
UIInterfaceOrientation currentOrientation = application.statusBarOrientation;
[application setStatusBarOrientation:currentOrientation animated:animated];
}
Good luck !
Since the interfaceOrientation is deprecated, the up to date swift 2.0 solution shall be:
let previewLayerConnection = self.previewLayer.connection
let orientation = AVCaptureVideoOrientation(rawValue: UIApplication.sharedApplication().statusBarOrientation.rawValue)
previewLayerConnection.videoOrientation = orientation!