I have this block of code that's used for setting up input/output for a video session. It works perfectly on iOS.
let videoOutput = AVCaptureVideoDataOutput()
guard let cameraDevice = device.device,
let captureDeviceInput = try? AVCaptureDeviceInput(device: cameraDevice),
self.canAddInput(captureDeviceInput),
self.canAddOutput(videoOutput) else {
log("FrameExtractor error: could not setup input or output.")
return
}
self.addInput(captureDeviceInput)
videoOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "sample buffer"))
self.addOutput(videoOutput)
guard let connection = videoOutput.connection(with: CAMERA_FEED_MEDIA_TYPE),
connection.isVideoOrientationSupported,
connection.isVideoMirroringSupported else {
log("FrameExtractor: configureSession: Cannot establish connection")
return
}
if let videoOrientation = getCameraOrientation(device: device) {
connection.videoOrientation = videoOrientation
}
// Mirroring the front camera normalizes the display output for iPad's front camera
connection.isVideoMirrored = position == .front
When run on Catalyst, however, I get a consistent crash at
self.addOutput(videoOutput) of
Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid parameter not satisfying: index>=0 && index<[_currentItems count]'
Related
I'm currently working on a camera app. Everything worked fine, but when I tried to change the constraints of the Vision View the log suddenly printed this error.
[warning]the specified colorspace format is not supported. falling back on libyuv.
I have no Idea where it comes from and what I should change. Below I'll past the relevant code where I set up the camera.
func initializeCameraSession() {
// Set up Values
//1: Create a new AV Session
// , xf , AVCaptureVideoDataOutputSampleBufferDelegate // Get camera devices
let devices = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInWideAngleCamera], mediaType: AVMediaType.video, position: .front).devices
//2: Select a capture device
avSession.sessionPreset = .low
do {
if let captureDevice = devices.first {
let captureDeviceInput = try AVCaptureDeviceInput(device: captureDevice)
}
avSession.beginConfiguration()
if avSession.canAddInput(captureDeviceInput) {
avSession.addInput(captureDeviceInput)
self.videoDeviceInput = captureDeviceInput
} else {
print("Couldn't add video device input to the session.")
avSession.commitConfiguration()
return
}
avSession.commitConfiguration()
}
} catch {
print(error.localizedDescription)
}
//3: Show output on a preview layer
let captureOutput = AVCaptureVideoDataOutput()
captureOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "videoQueue"))
captureOutput.videoSettings = [(kCVPixelBufferPixelFormatTypeKey as String) : NSNumber(value: kCVPixelFormatType_32BGRA )]
avSession.addOutput(captureOutput)
let previewLayer = AVCaptureVideoPreviewLayer(session: avSession)
previewLayer.videoGravity = AVLayerVideoGravity.resizeAspectFill
previewLayer.connection?.videoOrientation = .portrait
previewLayer.frame = visionView.bounds
visionView.layer.addSublayer(previewLayer)
view.bringSubviewToFront(visionView)
visionView.isHidden = true
visionView.alpha = 0.0
avSession.startRunning()
}
}
I've recently started running beta on my camera-based app. Everything is working as expected except on iPhone 6 devices.
The session starts on the back camera, and each time an iPhone 6 user switches to the front camera the app crashes. (And just to be really clear: no one on any other iPhone model is experiencing the issue.) I've gotten my hands on a 6 to test and can consistently reproduce the error, resulting in libc++abi.dylib: terminating with uncaught exception of type NSException.
I've tried starting the session on the front camera and it crashes immediately.
func initializeCamera() {
self.captureSession.sessionPreset = .hd1920x1080
let discovery = AVCaptureDevice.DiscoverySession.init(deviceTypes: [AVCaptureDevice.DeviceType.builtInWideAngleCamera],
mediaType: .video,
position: .unspecified) as AVCaptureDevice.DiscoverySession
for device in discovery.devices as [AVCaptureDevice] {
if device.hasMediaType(.video) {
if device.position == AVCaptureDevice.Position.front {
videoCaptureDevice = device
do {
try currentDeviceInput = AVCaptureDeviceInput(device: device)
} catch {
print("error: \(error.localizedDescription)")
}
}
}
}
if videoCaptureDevice != nil {
do {
let videoInput = try AVCaptureDeviceInput(device: videoCaptureDevice!)
captureSession.addInput(videoInput)
if let audioInput = AVCaptureDevice.default(for: .audio) {
try captureSession.addInput(AVCaptureDeviceInput(device: audioInput))
}
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
guard let previewLayer = previewLayer else { return }
cameraPreviewView.frame = cameraContainer.frame
cameraPreviewView.layer.addSublayer(previewLayer)
previewLayer.frame = cameraPreviewView.frame
previewLayer.videoGravity = AVLayerVideoGravity.resizeAspectFill
setVideoOrientation()
captureSession.addOutput(movieFileOutput)
if let movieFileOutputConnection = movieFileOutput.connection(with: .video) {
if movieFileOutputConnection.isVideoStabilizationSupported {
movieFileOutputConnection.preferredVideoStabilizationMode = .cinematic
}
}
captureSession.startRunning()
sessionIsReady(true)
} catch {
print("error: \(error.localizedDescription)")
}
}
}
func setVideoOrientation() {
if let connection = self.previewLayer?.connection {
if connection.isVideoOrientationSupported {
connection.videoOrientation = .portrait
previewLayer?.frame = cameraContainer.bounds
}
}
}
The crash is triggered at captureSession.addInput(videoInput). videoInput is not nil. The camera's orientation is locked to portrait.
Can anyone offer any insight? Please let me know if any additional code would be helpful. Thanks in advance.
captureSession.addInput(videoInput) is causing the crash.
So you should use canAddInput(_:) before to avoid the crash.
if captureSession.canAddInput(videoInput) {
captureSession.addInput(videoInput)
}
And in your case, captureSession.canAddInput(videoInput) == false with that iPhone 6.
Now, you are also doing self.captureSession.sessionPreset = .hd1920x1080
But according to WikiPedia, the iPhone 6 Front Camera hardware supports
camera 1.2 MP (1280×960 px max.), 720p video recording (30 fps). Doesn't seem to fit the 1920*1080 ("Full HD").
You could do this check what the "max" AVCaptureSession.Preset you can use.
func setSessionPreset(forDevice device: AVCaptureDevice) {
let videoPresets: [AVCaptureSession.Preset] = [.hd4K3840x2160, .hd1920x1080, .hd1280x720] //etc. Put them in order to "preferred" to "last preferred"
let preset = videoPresets.first(where: { device.supportsSessionPreset($0) }) ?? .hd1280x720
captureSession.sessionPreset = preset
}
All of the camera tweaks I am wanting to use are working except the zoomfactor. I am lost as to why this is happening...any ideas? The custom exposure and focus settings work fine. Did something change in iOS that im not aware of?
captureSession = AVCaptureSession()
captureSession?.sessionPreset = AVCaptureSessionPresetPhoto
stillImageOutput = AVCapturePhotoOutput()
let device = AVCaptureDevice.defaultDevice(withMediaType: AVMediaTypeVideo)
do{
do{
try device?.lockForConfiguration()
device?.setFocusModeLockedWithLensPosition(focusValue, completionHandler: {(time) -> Void in})
device?.setExposureModeCustomWithDuration(CMTimeMake(1, exposureValue), iso: ISOValue, completionHandler: {(time) -> Void in})
let zoomFactor:CGFloat = 16
device?.videoZoomFactor = zoomFactor
device?.unlockForConfiguration()
}catch{
print(error)
}
stillImageOutput.isHighResolutionCaptureEnabled = true
let input = try AVCaptureDeviceInput(device: device)
if(captureSession.canAddInput(input)){
captureSession.addInput(input)
if(captureSession.canAddOutput(stillImageOutput)){
captureSession.addOutput(stillImageOutput)
captureSession.startRunning()
let captureVideoLayer: AVCaptureVideoPreviewLayer = AVCaptureVideoPreviewLayer.init(session: captureSession)
captureVideoLayer.frame = self.previewView.bounds
captureVideoLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
self.previewView.layer.insertSublayer(captureVideoLayer, at: 0)
}
}
}catch{
print(error)
}
turns out i was just setting my device settings at the wrong point. If the lockconfig try block is moved below the capture session stuff it then works as intended.
Using Apple's example code for Photo & Video Acquisition (i.e. AVFoundation), I tried to change the device-zoom of my iPhone camera in code.
With the help of user2345335, I realised that the code-location where you place your zoom manipulation properties matters - and also, make sure you use device.lockForConfiguration() prior to any videoDevice manipulation ! Both are important (code-location and locking !!).
Here the Link and a screenshot to see the download-button where the original Apple example can be taken from :
(AVFoundation Apple Code example: Link)
Here is the code excerpt of the original Apple example with MY CODE THAT MANIPULATES THE ZOOM inserted at the correct spot :)
(Swift-4.2 / Xcode 10.0, iOS 11.0 SDK)
// Call this on the session queue.
private func configureSession() {
if setupResult != .success {
return
}
// ... missing original code (not important for this illustration)...
// Add video input.
do {
var defaultVideoDevice: AVCaptureDevice?
// ... missing original code (not important for this illustration)...
if session.canAddInput(videoDeviceInput) {
session.addInput(videoDeviceInput)
self.videoDeviceInput = videoDeviceInput
// ... missing original code (not important for this illustration)...
} else {
print("Could not add video device input to the session")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
// !!!!!!!!!!!!!!!! MY CODE THAT MANIPULATES THE ZOOM !!!!!!!!!!!!!!!!!!!!!!!!!!!!
// !!!!!!!!!!!!!!!! PLACE IT HERE AND ZOOM WILL WORK !!!!!!!!!!!!!!!!!!!!!!!!!!!!
guard let device = defaultVideoDevice else { return }
do {
try device.lockForConfiguration()
defer { device.unlockForConfiguration() }
device.videoZoomFactor = 10.0
} catch {
debugPrint(error)
}
// !!!!!!!!!!!!!! END OF MY CODE THAT MANIPULATES THE ZOOM !!!!!!!!!!!!!!!!!!!!!!!!!!
} catch {
print("Could not create video device input: \(error)")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
// Add audio input.
do {
let audioDevice = AVCaptureDevice.default(for: .audio)
let audioDeviceInput = try AVCaptureDeviceInput(device: audioDevice!)
if session.canAddInput(audioDeviceInput) {
session.addInput(audioDeviceInput)
} else {
print("Could not add audio device input to the session")
}
} catch {
print("Could not create audio device input: \(error)")
}
// Add photo output.
if session.canAddOutput(photoOutput) {
session.addOutput(photoOutput)
photoOutput.isHighResolutionCaptureEnabled = true
photoOutput.isLivePhotoCaptureEnabled = photoOutput.isLivePhotoCaptureSupported
photoOutput.isDepthDataDeliveryEnabled = photoOutput.isDepthDataDeliverySupported
livePhotoMode = photoOutput.isLivePhotoCaptureSupported ? .on : .off
depthDataDeliveryMode = photoOutput.isDepthDataDeliverySupported ? .on : .off
} else {
print("Could not add photo output to the session")
setupResult = .configurationFailed
session.commitConfiguration()
return
}
session.commitConfiguration()
}
I have faced an issue with zoom factor not working on some devices (usualy the newer ones with wide lens camera) when output.isDepthDataDeliveryEnabled was not set and therefore the default value was true . The problem was present only when AVCaptureSession.Preset was set to .photo.
I have a AVCaptureVideoPreviewLayerin my app that works well and is showing the same preview video as the camera app. I would like to implement the 2x zoom functionality of the camera app. How do I do this?
Basically I want my previewlayer to change the video feed to same scale as what you see in the camera app when you tap on the 1x icon to change it to 2x.
setting up preview layer
func startSession(){
captureSession = AVCaptureSession()
captureSession?.sessionPreset = AVCaptureSessionPresetPhoto
let backCamera = AVCaptureDevice.defaultDevice(withMediaType: AVMediaTypeVideo)
// Catch error using the do catch block
do {
let input = try AVCaptureDeviceInput(device: backCamera)
if (captureSession?.canAddInput(input) != nil){
captureSession?.addInput(input)
// Setup the preview layer
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer?.videoGravity = AVLayerVideoGravityResizeAspectFill
previewLayer?.connection.videoOrientation = AVCaptureVideoOrientation.portrait
tempImageView.layer.addSublayer(previewLayer!)
captureSession?.startRunning()
// Set up AVCaptureVideoDataOutput
let dataOutput = AVCaptureVideoDataOutput()
dataOutput.videoSettings = [(kCVPixelBufferPixelFormatTypeKey as NSString) : NSNumber(value: kCVPixelFormatType_32BGRA as UInt32)]
dataOutput.alwaysDiscardsLateVideoFrames = true
if (captureSession?.canAddOutput(dataOutput) == true) {
captureSession?.addOutput(dataOutput)
}
let queue = DispatchQueue(label: "com.bigbob.videoQueue")
dataOutput.setSampleBufferDelegate(self, queue: queue)
}
} catch _ {
print("Error setting up camera!")
}
Set the videoZoomFactor of your AVCaptureDevice.defaultDevice and the preview layer's zoom will follow suit. Note Swift 4 it is now called AVCaptureDevice.default.
do {
try backCamera?.lockForConfiguration()
let zoomFactor:CGFloat = 2
backCamera?.videoZoomFactor = zoomFactor
backCamera?.unlockForConfiguration()
} catch {
//Catch error from lockForConfiguration
}
Here's a bit of an updated answer that fist checks to make sure the zoom factor is available before you even attempt to set it. The will prevent possibly unneeded exception catches and you can adjust the zoom check and set easily with one variable.
if let captureDevice = AVCaptureDevice.default(for: AVMediaType.video) {
let zoomFactor : CGFloat = 2
if (captureDevice.maxAvailableVideoZoomFactor >= zoomFactor) {
try? captureDevice.lockForConfiguration()
captureDevice.videoZoomFactor = zoomFactor
captureDevice.unlockForConfiguration()
}
}
When I set the flashmode for my front camera and then call
let videoConnection = stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo)
stillImageOutput?.captureStillImageAsynchronouslyFromConnection(videoConnection, completionHandler: process)
I get the following error message:
error while capturing still image: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSUnderlyingError=0x12eeb7200 {Error Domain=NSOSStatusErrorDomain Code=-16800 "(null)"}, NSLocalizedFailureReason=An unknown error occurred (-16800), NSLocalizedDescription=The operation could not be completed}
If I don't set the camera's flashMode and then call:
let videoConnection = stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo)
stillImageOutput?.captureStillImageAsynchronouslyFromConnection(videoConnection, completionHandler: process)
The front camera takes a picture and doesn't throw the error. So I wonder, does a front-camera flash exist for iphones. It should considering that snapchat has one. And the default camera app on an iPhone has a front camera flash. So I'm not entirely sure what's going on. Currently, this is how I set up my camera:
func getCameraStreamLayer() -> CALayer? {
captureSession = AVCaptureSession()
captureSession!.sessionPreset = AVCaptureSessionPresetPhoto
currentCamera = AVCaptureDevice.defaultDeviceWithMediaType(AVMediaTypeVideo)
stillImageOutput = AVCaptureStillImageOutput()
stillImageOutput!.outputSettings = [ AVVideoCodecKey: AVVideoCodecJPEG ]
if let input = try? AVCaptureDeviceInput(device: currentCamera) as AVCaptureDeviceInput{
if captureSession!.canAddInput(input) && captureSession!.canAddOutput(stillImageOutput) {
captureSession!.addInput(input)
captureSession!.addOutput(stillImageOutput)
}
}
return AVCaptureVideoPreviewLayer(session: captureSession)
}
func toggleFlash() {
flash = !flash
if flash {
for case let (device as AVCaptureDevice) in AVCaptureDevice.devices() {
if device.hasFlash && device.flashAvailable {
if device.isFlashModeSupported(.On) {
do {
try device.lockForConfiguration()
device.flashMode = .On
device.unlockForConfiguration()
} catch {
print("Something went wrong")
}
}
}
}
}else {//turn off flash
}
}
func photograph(process: (CMSampleBuffer!,NSError!)->()) {
let videoConnection = stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo)
stillImageOutput?.captureStillImageAsynchronouslyFromConnection(videoConnection, completionHandler: process)
}
func flipCamera() {
guard let session = captureSession where session.running == true else {
return
}
session.beginConfiguration()
let currentCameraInput = session.inputs[0] as! AVCaptureDeviceInput
session.removeInput(currentCameraInput)
let newCamera = {
let devices = AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo)
for case let device as AVCaptureDevice in devices {
if(device.position == .Front && currentCameraInput.device.position == .Back){
return device
}
if(device.position == .Back && currentCameraInput.device.position == .Front){
return device
}
}
return nil
}() as AVCaptureDevice?
currentCamera = newCamera!
if let newVideoInput = try? AVCaptureDeviceInput(device: newCamera) {
captureSession?.addInput(newVideoInput)
}
captureSession?.commitConfiguration()
}
I'm not sure what I should do. I've tried to create a new capture session and then lock and then set the flashMode for the camera. I still get the same error.
iPhone 6 does not have a front facing flash camera, however the iPhone 6s and up does.
There are "hack" solutions in the app store that flash the screen brightly to generate light in front facing mode, but there's no actual flash.