IOS camera zoom not working swift - xcode - ios

I have iPhone 13 pro with iOS version 15.5. I am using XCode version 13.3.1.
I am trying to achieve camera zoom with the AVFoundation.
Below is the code I have tried out. The camera device selected is the .builtInTripleCamera. I would like to take macro images. I am trying out different zoom factors like 1.0, 3.0 and 15.0. But the camera preview is not changing. I am not sure whether this is the current method to obtain zoomed-in camera preview as well as the output. Did anybody manage to correctly set the camera zoom factor with AVFoundation?
private let captureSession = AVCaptureSession()
private lazy var previewLayer = AVCaptureVideoPreviewLayer(session: self.captureSession)
private let videoDataOutput = AVCaptureVideoDataOutput()
private var cameraDevice: AVCaptureDevice?
override func viewDidLoad() {
super.viewDidLoad()
self.setCameraInput()
self.setCameraOutput()
self.showCameraFeed()
self.captureSession.beginConfiguration()
self.captureSession.sessionPreset = AVCaptureSession.Preset.high;
self.captureSession.commitConfiguration()
self.captureSession.startRunning()
}
private func showCameraFeed() {
self.previewLayer.videoGravity = AVLayerVideoGravity.resizeAspectFill
self.view.layer.insertSublayer(self.previewLayer, at: 1)
self.previewLayer.frame = self.view.frame
// self.view.addGestureRecognizer(pgr)
}
private func setCameraInput() {
let deviceTypes: [AVCaptureDevice.DeviceType]
if #available(iOS 13, *) {
deviceTypes = [.builtInTripleCamera, .builtInDualWideCamera, .builtInDualCamera, .builtInWideAngleCamera]
} else {
deviceTypes = [.builtInDualCamera, .builtInWideAngleCamera]
}
let session = AVCaptureDevice.DiscoverySession(
deviceTypes: deviceTypes,
mediaType: .video,
position: .back
)
guard let device = session.devices.first else {
fatalError("No back camera device found.")
}
cameraDevice = device
addInputToSession(device: cameraDevice!, session: self.captureSession)
}
private func addInputToSession(device:AVCaptureDevice,session:AVCaptureSession){
let cameraInput = try! AVCaptureDeviceInput(device: device)
if(session.canAddInput(cameraInput)){
do{
try device.lockForConfiguration()
cameraInput.device.videoZoomFactor = 15
session.addInput(cameraInput)
device.unlockForConfiguration()
}catch{
print(error)
}
}
}
private func setCameraOutput() {
self.videoDataOutput.videoSettings = [(kCVPixelBufferPixelFormatTypeKey as NSString) : NSNumber(value: kCVPixelFormatType_32BGRA)] as [String : Any]
self.videoDataOutput.alwaysDiscardsLateVideoFrames = true
self.videoDataOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "camera_frame_processing_queue"))
self.captureSession.addOutput(self.videoDataOutput)
guard let connection = self.videoDataOutput.connection(with: AVMediaType.video),
connection.isVideoOrientationSupported else { return }
connection.videoOrientation = .portrait
}

Handle camera zoom with pinch recognizer.
cameraDevice.videoZoomFactor
and set it to the recognizer on .began state like this
#objc private func viewPinched(recognizer: UIPinchGestureRecognizer) {
switch recognizer.state {
case .began:
recognizer.scale = cameraDevice.videoZoomFactor
case .changed:
let scale = recognizer.scale
do {
try cameraDevice.lockForConfiguration()
cameraDevice.videoZoomFactor = max(cameraDevice.minAvailableVideoZoomFactor, min(scale, cameraDevice.maxAvailableVideoZoomFactor))
cameraDevice.unlockForConfiguration()
}
catch {
print(error)
}
default:
break
}
}

Related

AVFoundation's ultra-wide-angle camera behaves differently than the default apple camera

Thanks for reading.
I am creating a custom camera using AVFoundation.
When I maximize the wide angle with the ultra wide angle camera (when videoZoomFactor is minimized), the wide angle field of view is narrower compared to the apple default camera.
Looking at the metadata from the album, the focal length is 13mm for the apple default camera, while it is 16mm for the one I created. Below is an excerpt of the code.
Camera Settings
if let captureDevice = AVCaptureDevice.default(
.builtInTripleCamera,
for: .video,
position: .back
) {
self.captureDevice = captureDevice
} else if let captureDevice = AVCaptureDevice.default(
.builtInDualWideCamera,
for: .video,
position: .back
) {
self.captureDevice = captureDevice
} else if let captureDevice = AVCaptureDevice.default(
.builtInWideAngleCamera,
for: .video,
position: .back
) {
self.captureDevice = captureDevice
}
do {
let input = try AVCaptureDeviceInput(device: captureDevice)
let videoDataOutput = AVCaptureVideoDataOutput()
// Omitted
photoOutput = AVCapturePhotoOutput()
guard let photoOutput = photoOutput else { return }
photoOutput.isHighResolutionCaptureEnabled = true
session.sessionPreset = .photo
// Omitted
} catch {
}
for connection in session.connections {
connection.preferredVideoStabilizationMode = .cinematicExtended
}
zoom function
func zoom(zoomFactor: CGFloat, ramping: Bool = false) {
do {
try captureDevice?.lockForConfiguration()
self.zoomFactor = zoomFactor
if ramping {
captureDevice?.ramp(toVideoZoomFactor: zoomFactor, withRate: 10.0)
} else {
captureDevice?.videoZoomFactor = zoomFactor
}
captureDevice?.unlockForConfiguration()
} catch {
errorReportingService.reportError(error: error)
}
}
Test devices: iPhone 11, 12mini
Thanks for reading this far. I want to make it as wide angle as the apple default camera!
This app allows for a wider angle than the one I created. So I believe there is a way.

Anyone know how to use Apple's vision framework for real-time text recognition?

I can't seem to find a way to not use the document scanner, and supplement it with AVFoundation instead. I'm trying to create a feature where the user can click a button, scan text, and then save that to some textview w/o having the user click the camera button, keep scan, save, etc.
I've got it to work with object detection, but I can't get it to work for text-recognition. So, is there any way to use Apple's vision framework for real-time text recognition? Any help would be much appreciated
For performance reasons, I'd prefer to not convert the CMSampleBuffer to a UIImage, and would instead use the following to create an AVCaptureVideoPreviewLayer for live video:
class CameraFeedView: UIView {
private var previewLayer: AVCaptureVideoPreviewLayer!
override class var layerClass: AnyClass {
return AVCaptureVideoPreviewLayer.self
}
init(frame: CGRect, session: AVCaptureSession, videoOrientation: AVCaptureVideoOrientation) {
super.init(frame: frame)
previewLayer = layer as? AVCaptureVideoPreviewLayer
previewLayer.session = session
previewLayer.videoGravity = .resizeAspect
previewLayer.connection?.videoOrientation = videoOrientation
}
required init?(coder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
Once you have this, you can work on the live video data using Vision:
class CameraViewController: AVCaptureVideoDataOutputSampleBufferDelegate {
private let videoDataOutputQueue = DispatchQueue(label: "CameraFeedDataOutput", qos: .userInitiated,
attributes: [], autoreleaseFrequency: .workItem)
private var drawingView: UILabel = {
let view = UILabel(frame: UIScreen.main.bounds)
view.font = UIFont.boldSystemFont(ofSize: 30.0)
view.textColor = .red
view.translatesAutoresizingMaskIntoConstraints = false
return view
}()
private var cameraFeedSession: AVCaptureSession?
private var cameraFeedView: CameraFeedView! //Wrap
override func viewDidLoad() {
super.viewDidLoad()
do {
try setupAVSession()
} catch {
print("setup av session failed")
}
}
func setupAVSession() throws {
// Create device discovery session for a wide angle camera
let wideAngle = AVCaptureDevice.DeviceType.builtInWideAngleCamera
let discoverySession = AVCaptureDevice.DiscoverySession(deviceTypes: [wideAngle], mediaType: .video, position: .back)
// Select a video device, make an input
guard let videoDevice = discoverySession.devices.first else {
print("Could not find a wide angle camera device.")
}
guard let deviceInput = try? AVCaptureDeviceInput(device: videoDevice) else {
print("Could not create video device input.")
}
let session = AVCaptureSession()
session.beginConfiguration()
// We prefer a 1080p video capture but if camera cannot provide it then fall back to highest possible quality
if videoDevice.supportsSessionPreset(.hd1920x1080) {
session.sessionPreset = .hd1920x1080
} else {
session.sessionPreset = .high
}
// Add a video input
guard session.canAddInput(deviceInput) else {
print("Could not add video device input to the session")
}
session.addInput(deviceInput)
let dataOutput = AVCaptureVideoDataOutput()
if session.canAddOutput(dataOutput) {
session.addOutput(dataOutput)
// Add a video data output
dataOutput.alwaysDiscardsLateVideoFrames = true
dataOutput.videoSettings = [
String(kCVPixelBufferPixelFormatTypeKey): Int(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange)
]
dataOutput.setSampleBufferDelegate(self, queue: videoDataOutputQueue)
} else {
print("Could not add video data output to the session")
}
let captureConnection = dataOutput.connection(with: .video)
captureConnection?.preferredVideoStabilizationMode = .standard
captureConnection?.videoOrientation = .portrait
// Always process the frames
captureConnection?.isEnabled = true
session.commitConfiguration()
cameraFeedSession = session
// Get the interface orientaion from window scene to set proper video orientation on capture connection.
let videoOrientation: AVCaptureVideoOrientation
switch view.window?.windowScene?.interfaceOrientation {
case .landscapeRight:
videoOrientation = .landscapeRight
default:
videoOrientation = .portrait
}
// Create and setup video feed view
cameraFeedView = CameraFeedView(frame: view.bounds, session: session, videoOrientation: videoOrientation)
setupVideoOutputView(cameraFeedView)
cameraFeedSession?.startRunning()
}
The key functions to implement once you've got an AVCaptureSession set up are the delegate and request handler:
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
let requestHandler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer, orientation: .down)
let request = VNRecognizeTextRequest(completionHandler: textDetectHandler)
do {
// Perform the text-detection request.
try requestHandler.perform([request])
} catch {
print("Unable to perform the request: \(error).")
}
}
func textDetectHandler(request: VNRequest, error: Error?) {
guard let observations =
request.results as? [VNRecognizedTextObservation] else { return }
// Process each observation to find the recognized body pose points.
let recognizedStrings = observations.compactMap { observation in
// Return the string of the top VNRecognizedText instance.
return observation.topCandidates(1).first?.string
}
DispatchQueue.main.async {
self.drawingView.text = recognizedStrings.first
}
}
}
Note, you will probably want to process each of the recognizedStrings in order to choose the one with the highest confidence, but this is a proof of concept. You could also add a bounding box, and the docs have an example of that.

Preview doesn't show up (AVCaptureVideoPreviewLayer)

I'm trying to first just get my app to show the camera preview of the wide angle camera.
But what I do it just doesn't show up. (the app has video and audio permission and is being tested on an iPhone 7 Plus)
Here's my session configuration code:
func configureSession() throws {
session.beginConfiguration()
// configure inputs
let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back)
let audioDevice = AVCaptureDevice.default(.builtInMicrophone, for: .audio, position: .unspecified)
guard let videoInput = try? AVCaptureDeviceInput(device: videoDevice!), let audioInput = try? AVCaptureDeviceInput(device: audioDevice!), session.canAddInput(videoInput), session.canAddInput(audioInput) else {
throw CameraError.configurationFailed
}
// configure outputs
let output = AVCaptureMovieFileOutput()
let previewOutput = AVCaptureVideoDataOutput()
previewOutput.alwaysDiscardsLateVideoFrames = true
previewOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32BGRA)]
guard session.canAddOutput(output) && session.canAddOutput(previewOutput) else {
throw CameraError.configurationFailed
}
session.sessionPreset = .high
session.addOutput(output)
session.addOutput(previewOutput)
session.commitConfiguration()
}
and this is the code for the previewLayer:
// adds given view as previewView to the session
func configureAsPreview(_ view: UIView) {
let previewLayer = AVCaptureVideoPreviewLayer(session: session)
view.layer.addSublayer(previewLayer)
previewLayer.frame = view.layer.frame
}
and it is being called in viewDidLoad:
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
if cameraManager.hasPermissionForVideo {
try! cameraManager.configureSession()
cameraManager.configureAsPreview(self.previewView)
cameraManager.startSession()
} else {
performSegue(withIdentifier: "presentPermissions", sender: nil)
}
}
What am I doing wrong?
I really can't figure it out and it's probably just something stupidly small.
Thanks.
Try update layer frame when needed, and use bounds instead of frame. Layers dont support autoresizing, you need to manage it manually.
private var __previewLayer = AVCaptureVideoPreviewLayer? = nil
func configureAsPreview(_ view: UIView) {
let previewLayer = AVCaptureVideoPreviewLayer(session: session)
self.view.layer.addSublayer(previewLayer)
previewLayer.frame = self.view.bounds
self.__previewLayer = previewLayer
}
override viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
self.__previewLayer?.frame = self.view.bounds
}

How can i record an mp4 video in mpeg4 container with AVCaptureSession with swift (IOS 8)

I will answer my own question to share my experience, since there is no complete working code on the internet.
IOS devices usually record videos in .mov files with quicktime format. Even the output video has AVC baseline video codec and AAC audio codec, the resulting file will be in quicktime container. And those videos may not play in android devices. Apple has Avfoundation classes like AvCaptureSession and AVCaptureMovieFileOutput but they do not directly support mp4 file output. How can i record an actual mp4 video in mpeg4 container with swift and IOS 8 support?
First things first: This may not be the best solution, but this is a complete solution.
The code below captures video and audio with AvCaptureSession and converts it into mpeg4 with AvExportSession. There is also zoom in, zoom out and switch camera functionality and permission checking. You can record in 480p or 720p. You can also set minimum and maximum frame rates to create smaller videos. Hope this helps as a complete guide.
Note: There are keys to add to info.plist to ask for camera and photos library permission:
<key>NSCameraUsageDescription</key>
<string>Yo, this is a cam app.</string>
<key>NSPhotoLibraryUsageDescription</key>
<string>Yo, i need to access your photos.</string>
<key>NSMicrophoneUsageDescription</key>
<string>Yo, i can't hear you</string>
And the code:
import UIKit
import Photos
import AVFoundation
class VideoAct: UIViewController, AVCaptureFileOutputRecordingDelegate
{
let captureSession : AVCaptureSession = AVCaptureSession()
var captureDevice : AVCaptureDevice!
var microphone : AVCaptureDevice!
var previewLayer : AVCaptureVideoPreviewLayer!
let videoFileOutput : AVCaptureMovieFileOutput = AVCaptureMovieFileOutput()
var duration : Int = 30
var v_path : URL = URL(fileURLWithPath: "")
var my_timer : Timer = Timer()
var cameraFront : Bool = false
var cameras_number : Int = 0
var max_zoom : CGFloat = 76
var devices : [AVCaptureDevice] = []
var captureInput : AVCaptureDeviceInput = AVCaptureDeviceInput()
var micInput : AVCaptureDeviceInput = AVCaptureDeviceInput()
#IBOutlet weak var cameraView: UIView!
override func viewDidLoad()
{
super.viewDidLoad()
if (check_permissions())
{
initialize()
}
else
{
AVCaptureDevice.requestAccess(forMediaType: AVMediaTypeVideo, completionHandler: { (granted) in
if (granted)
{
self.initialize()
}
else
{
self.dismiss(animated: true, completion: nil)
}
})
}
}
func check_permissions() -> Bool
{
return AVCaptureDevice.authorizationStatus(forMediaType: AVMediaTypeVideo) == AVAuthorizationStatus.authorized
}
#available(iOS 4.0, *)
func capture(_ captureOutput: AVCaptureFileOutput!, didFinishRecordingToOutputFileAt outputFileURL: URL!, fromConnections connections: [Any]!, error: Error!)
{
//you can implement stopvideoaction here if you want
}
func initialize()
{
let directory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
v_path = directory.appendingPathComponent("temp_video.mp4")
// we just set the extension .mp4 but
// actually it is a mov file with QT container !! May not play in Android devices.
// it will be ceonverted
self.duration = 30
devices = AVCaptureDevice.devices() as! [AVCaptureDevice]
for device in devices
{
if (device.hasMediaType(AVMediaTypeVideo))
{
if (device.position == AVCaptureDevicePosition.back)
{
captureDevice = device as AVCaptureDevice
}
if (device.position == AVCaptureDevicePosition.front)
{
cameras_number = 2
}
}
if (device.hasMediaType(AVMediaTypeAudio))
{
microphone = device as AVCaptureDevice
}
}
if (cameras_number == 1)
{
//only 1 camera available
btnSwitchCamera.isHidden = true
}
if captureDevice != nil
{
beginSession()
}
max_zoom = captureDevice.activeFormat.videoMaxZoomFactor
}
func beginSession()
{
if (captureSession.isRunning)
{
captureSession.stopRunning()
}
do
{
try captureInput = AVCaptureDeviceInput(device: captureDevice)
try micInput = AVCaptureDeviceInput(device: microphone)
try captureDevice.lockForConfiguration()
}
catch
{
print("errorrrrrrrrrrr \(error)")
}
// beginconfig before adding input and setting settings
captureSession.beginConfiguration()
captureSession.addInput(captureInput)
captureSession.addInput(micInput)
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer.connection.videoOrientation = AVCaptureVideoOrientation.init(rawValue: UIDevice.current.orientation.rawValue)!
if (previewLayer.connection.isVideoStabilizationSupported)
{
previewLayer.connection.preferredVideoStabilizationMode = AVCaptureVideoStabilizationMode.auto
}
if (captureDevice.isSmoothAutoFocusSupported)
{
captureDevice.isSmoothAutoFocusEnabled = false
}
if (captureDevice.isFocusModeSupported(AVCaptureFocusMode.continuousAutoFocus))
{
captureDevice.focusMode = .continuousAutoFocus
}
set_preview_size_thing()
set_quality_thing()
if (captureDevice.isLowLightBoostSupported)
{
captureDevice.automaticallyEnablesLowLightBoostWhenAvailable = true
}
if (cameraView.layer.sublayers?[0] is AVCaptureVideoPreviewLayer)
{
//to prevent previewlayers stacking on every camera switch
cameraView.layer.sublayers?.remove(at: 0)
}
cameraView.layer.insertSublayer(previewLayer, at: 0)
previewLayer?.frame = cameraView.layer.frame
captureSession.commitConfiguration()
captureSession.startRunning()
}
func duration_thing()
{
// there is a textview to write remaining time left
self.duration = self.duration - 1
timerTextView.text = "remaining seconds: \(self.duration)"
timerTextView.sizeToFit()
if (self.duration == 0)
{
my_timer.invalidate()
stopVideoAction()
}
}
func switch_cam()
{
captureSession.removeInput(captureInput)
captureSession.removeInput(micInput)
cameraFront = !cameraFront
// capturedevice will be locked again
captureDevice.unlockForConfiguration()
for device in devices
{
if (device.hasMediaType(AVMediaTypeVideo))
{
if (device.position == AVCaptureDevicePosition.back && !cameraFront)
{
captureDevice = device as AVCaptureDevice
}
else if (device.position == AVCaptureDevicePosition.front && cameraFront)
{
captureDevice = device as AVCaptureDevice
}
}
}
beginSession()
}
func zoom_in()
{
// 10x zoom would be enough
if (captureDevice.videoZoomFactor * 1.5 < 10)
{
captureDevice.videoZoomFactor = captureDevice.videoZoomFactor * 1.5
}
else
{
captureDevice.videoZoomFactor = 10
}
}
func zoom_out()
{
if (captureDevice.videoZoomFactor * 0.67 > 1)
{
captureDevice.videoZoomFactor = captureDevice.videoZoomFactor * 0.67
}
else
{
captureDevice.videoZoomFactor = 1
}
}
func set_quality_thing()
{
// there is a switch in the screen (30-30 fps high quality or 15-23 fps normal quality)
// you may not have to do this because export session also has some presets and a property called “optimizefornetwork” or something. But it would be better to make sure the output file is not huge with unnecessary 90 fps video
captureDevice.activeVideoMinFrameDuration = CMTimeMake(1, switch_quality.isOn ? 30 : 15)
captureDevice.activeVideoMaxFrameDuration = CMTimeMake(1, switch_quality.isOn ? 30 : 23)
}
func set_preview_size_thing()
{
//there is a switch for resolution (720p or 480p)
captureSession.sessionPreset = switch_res.isOn ? AVCaptureSessionPreset1280x720 : AVCaptureSessionPreset640x480
//this for loop is probably unnecessary and ridiculous but you can make sure you are using the right format
for some_format in captureDevice.formats as! [AVCaptureDeviceFormat]
{
let some_desc : String = String(describing: some_format)
if (switch_res.isOn)
{
if (some_desc.contains("1280x") && some_desc.contains("720") && some_desc.contains("420v") && some_desc.contains("30 fps"))
{
captureDevice.activeFormat = some_format
break
}
}
else
{
if (some_desc.contains("640x") && some_desc.contains("480") && some_desc.contains("420v"))
{
captureDevice.activeFormat = some_format
break
}
}
}
}
func takeVideoAction()
{
// movieFragmentInterval is important !! or you may end up with a video without audio
videoFileOutput.movieFragmentInterval = kCMTimeInvalid
captureSession.addOutput(videoFileOutput)
(videoFileOutput.connections.first as! AVCaptureConnection).videoOrientation = returnedOrientation()
videoFileOutput.maxRecordedDuration = CMTime(seconds: Double(self.duration), preferredTimescale: 1)
videoFileOutput.startRecording(toOutputFileURL: v_path, recordingDelegate: self)
//timer will tell the remaining time
my_timer = Timer.scheduledTimer(timeInterval: 1, target: self, selector: #selector(duration_thing), userInfo: nil, repeats: true)
}
func stopVideoAction()
{
captureDevice.unlockForConfiguration()
videoFileOutput.stopRecording()
captureSession.stopRunning()
// turn temp_video into an .mpeg4 (mp4) video
let directory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
let avAsset = AVURLAsset(url: v_path, options: nil)
// there are other presets than AVAssetExportPresetPassthrough
let exportSession = AVAssetExportSession(asset: avAsset, presetName: AVAssetExportPresetPassthrough)!
exportSession.outputURL = directory.appendingPathComponent("main_video.mp4")
// now it is actually in an mpeg4 container
exportSession.outputFileType = AVFileTypeMPEG4
let start = CMTimeMakeWithSeconds(0.0, 0)
let range = CMTimeRangeMake(start, avAsset.duration)
exportSession.timeRange = range
exportSession.exportAsynchronously(completionHandler: {
if (exportSession.status == AVAssetExportSessionStatus.completed)
{
// you don’t need temp video after exporting main_video
do
{
try FileManager.default.removeItem(atPath: self.v_path.path)
}
catch
{
}
// v_path is now points to mp4 main_video
self.v_path = directory.appendingPathComponent("main_video.mp4")
self.performSegue(withIdentifier: "ShareVideoController", sender: nil)
}
})
}
func btn_capture_click_listener()
{
if (videoFileOutput.isRecording)
{
stopVideoAction()
}
else
{
takeVideoAction()
}
}
func returnedOrientation() -> AVCaptureVideoOrientation
{
var videoOrientation: AVCaptureVideoOrientation!
let orientation = UIDevice.current.orientation
switch orientation
{
case .landscapeLeft:
videoOrientation = .landscapeRight
case .landscapeRight:
videoOrientation = .landscapeLeft
default:
videoOrientation = .landscapeLeft
}
return videoOrientation
}
override func prepare(for segue: UIStoryboardSegue, sender: Any?)
{
if (segue.identifier == "ShareVideoController")
{
//to make it visible in the camera roll (main_video.mp4)
PHPhotoLibrary.shared().performChanges({PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: self.v_path)}) { completed, error in}
let destVC : ShareVideoController = segue.destination as! ShareVideoController
// use the path in other screen to upload it or whatever
destVC.videoFilePath = v_path
// bla bla
}
}
override var supportedInterfaceOrientations: UIInterfaceOrientationMask
{
// screen will always be in landscape (remove this override if you want)
return .landscape
}
}

Swift IOS Record Video and Audio with AVFoundation

I was able to successfully grab the recorded video by following this question
here
Basically
Inherit from AVCaptureFileOutputRecordingDelegate prototype
Loop through available devices
Creating a session with the camera
Start Recording
Stop Recording
Get the Record video by implementing above prototype's method
But the file doesn't comes with the audio.
According to this question, i have to record audio separately and merge the video and audio using mentioned classes
But i have no idea how to implement video and audio recording at the same time.
for device in devices {
// Make sure this particular device supports video
if (device.hasMediaType(AVMediaTypeVideo)) {
// Finally check the position and confirm we've got the back camera
if(device.position == AVCaptureDevicePosition.Back) {
captureDevice = device as? AVCaptureDevice
if captureDevice != nil {
print("Capture device found")
beginSession()
}
}
}
}
in this loop only available device types are .Front and .Back
Following is the way to record video with audio using AVFoundation framework. The steps are:
1. Prepare the session:
self.captureSession = AVCaptureSession()
2. Prepare available video and audio devices:
let session = AVCaptureDevice.DiscoverySession.init(deviceTypes:[.builtInWideAngleCamera, .builtInMicrophone], mediaType: AVMediaType.video, position: AVCaptureDevice.Position.unspecified)
let cameras = (session.devices.compactMap{$0})
for camera in cameras {
if camera.position == .front {
self.frontCamera = camera
}
if camera.position == .back {
self.rearCamera = camera
try camera.lockForConfiguration()
camera.focusMode = .continuousAutoFocus
camera.unlockForConfiguration()
}
}
3. Prepare session inputs:
guard let captureSession = self.captureSession else {
throw CameraControllerError.captureSessionIsMissing
}
if let rearCamera = self.rearCamera {
self.rearCameraInput = try AVCaptureDeviceInput(device: rearCamera)
if captureSession.canAddInput(self.rearCameraInput!) {
captureSession.addInput(self.rearCameraInput!)
self.currentCameraPosition = .rear
} else {
throw CameraControllerError.inputsAreInvalid
}
} else if let frontCamera = self.frontCamera {
self.frontCameraInput = try AVCaptureDeviceInput(device: frontCamera)
if captureSession.canAddInput(self.frontCameraInput!) {
captureSession.addInput(self.frontCameraInput!)
self.currentCameraPosition = .front
} else {
throw CameraControllerError.inputsAreInvalid
}
} else {
throw CameraControllerError.noCamerasAvailable
}
// Add audio input
if let audioDevice = self.audioDevice {
self.audioInput = try AVCaptureDeviceInput(device: audioDevice)
if captureSession.canAddInput(self.audioInput!) {
captureSession.addInput(self.audioInput!)
} else {
throw CameraControllerError.inputsAreInvalid
}
}
4. Prepare output:
self.videoOutput = AVCaptureMovieFileOutput()
if captureSession.canAddOutput(self.videoOutput!) {
captureSession.addOutput(self.videoOutput!)
}
captureSession.startRunning()
5. Start recording:
func recordVideo(completion: #escaping (URL?, Error?) -> Void) {
guard let captureSession = self.captureSession, captureSession.isRunning else {
completion(nil, CameraControllerError.captureSessionIsMissing)
return
}
let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
let fileUrl = paths[0].appendingPathComponent("output.mp4")
try? FileManager.default.removeItem(at: fileUrl)
videoOutput!.startRecording(to: fileUrl, recordingDelegate: self)
self.videoRecordCompletionBlock = completion
}
6. Stop recording:
func stopRecording(completion: #escaping (Error?) -> Void) {
guard let captureSession = self.captureSession, captureSession.isRunning else {
completion(CameraControllerError.captureSessionIsMissing)
return
}
self.videoOutput?.stopRecording()
}
7. Implement the delegate:
func fileOutput(_ output: AVCaptureFileOutput, didFinishRecordingTo outputFileURL: URL, from connections: [AVCaptureConnection], error: Error?) {
if error == nil {
//do something
} else {
//do something
}
}
I took idea from here: https://www.appcoda.com/avfoundation-swift-guide/
Here is the complete project https://github.com/rubaiyat6370/iOS-Tutorial/
Found the answer, This answer goes with this code
It can simply done by
declare another capture device variable
loop through devices and initialize camera and audio capture device variable
add audio input to session
code
var captureDevice : AVCaptureDevice?
var captureAudio :AVCaptureDevice?
Loop through devices and Initialize capture devices
var captureDeviceVideoFound: Bool = false
var captureDeviceAudioFound:Bool = false
// Loop through all the capture devices on this phone
for device in devices {
// Make sure this particular device supports video
if (device.hasMediaType(AVMediaTypeVideo)) {
// Finally check the position and confirm we've got the front camera
if(device.position == AVCaptureDevicePosition.Front) {
captureDevice = device as? AVCaptureDevice //initialize video
if captureDevice != nil {
print("Capture device found")
captureDeviceVideoFound = true;
}
}
}
if(device.hasMediaType(AVMediaTypeAudio)){
print("Capture device audio init")
captureAudio = device as? AVCaptureDevice //initialize audio
captureDeviceAudioFound = true
}
}
if(captureDeviceAudioFound && captureDeviceVideoFound){
beginSession()
}
Inside Session
try captureSession.addInput(AVCaptureDeviceInput(device: captureDevice))
try captureSession.addInput(AVCaptureDeviceInput(device: captureAudio))
This will output the video file with audio. no need to merge audio or do anything.
This apples documentation helps
Followed the answer from #Mumu but it didn't work for me because of the call to AVCaptureDevice.DiscoverySession.init that was returning video devices only.
Here is my version that works on iOS 14, Swift 5:
var captureSession: AVCaptureSession? = nil
var camera: AVCaptureDevice? = nil
var microphone: AVCaptureDevice? = nil
var videoOutput: AVCaptureFileOutput? = nil
var previewLayer: AVCaptureVideoPreviewLayer? = nil
func findDevices() {
camera = nil
microphone = nil
//Search for video media type and we need back camera only
let session = AVCaptureDevice.DiscoverySession.init(deviceTypes:[.builtInWideAngleCamera],
mediaType: AVMediaType.video, position: AVCaptureDevice.Position.back)
var devices = (session.devices.compactMap{$0})
//Search for microphone
let asession = AVCaptureDevice.DiscoverySession.init(deviceTypes:[.builtInMicrophone],
mediaType: AVMediaType.audio, position: AVCaptureDevice.Position.unspecified)
//Combine all devices into one list
devices.append(contentsOf: asession.devices.compactMap{$0})
for device in devices {
if device.position == .back {
do {
try device.lockForConfiguration()
device.focusMode = .continuousAutoFocus
device.flashMode = .off
device.whiteBalanceMode = .continuousAutoWhiteBalance
device.unlockForConfiguration()
camera = device
} catch {
}
}
if device.hasMediaType(.audio) {
microphone = device
}
}
}
func initVideoRecorder()->Bool {
captureSession = AVCaptureSession()
guard let captureSession = captureSession else {return false}
captureSession.sessionPreset = .hd4K3840x2160
findDevices()
guard let camera = camera else { return false}
do {
let cameraInput = try AVCaptureDeviceInput(device: camera)
captureSession.addInput(cameraInput)
} catch {
self.camera = nil
return false
}
if let audio = microphone {
do {
let audioInput = try AVCaptureDeviceInput(device: audio)
captureSession.addInput(audioInput)
} catch {
}
}
videoOutput = AVCaptureMovieFileOutput()
if captureSession.canAddOutput(videoOutput!) {
captureSession.addOutput(videoOutput!)
captureSession.startRunning()
videoOutput?.connection(with: .video)?.videoOrientation = .landscapeRight
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer?.videoGravity = .resizeAspect
previewLayer?.connection?.videoOrientation = .landscapeRight
return true
}
return false
}
func startRecording()->Bool {
guard let captureSession = captureSession, captureSession.isRunning else {return false}
let paths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
let fileUrl = paths[0].appendingPathComponent(getVideoName())
try? FileManager.default.removeItem(at: fileUrl)
videoOutput?.startRecording(to: fileUrl, recordingDelegate: self)
return true
}
I had this problem also, but when I grouped adding the video input and the sound input after, the audio worked. This is my code for adding the inputs.
if (cameraSession.canAddInput(deviceInput) == true && cameraSession.canAddInput(audioDeviceInput) == true) {//detects if devices can be added
cameraSession.addInput(deviceInput)//adds video
cameraSession.addInput(audioDeviceInput)//adds audio
}
Also I found you have to have video input first or else there won't be audio. I originally had them in two if statements, but I found putting them in one lets video and audio be recorded together. Hope this helps.
Record Video With Audio
//Get Video Device
if let devices = AVCaptureDevice.devices(withMediaType: AVMediaTypeVideo) as? [AVCaptureDevice] {
for device in devices {
if device.hasMediaType(AVMediaTypeVideo) {
if device.position == AVCaptureDevicePosition.back {
videoCaptureDevice = device
}
}
}
if videoCaptureDevice != nil {
do {
// Add Video Input
try self.captureSession.addInput(AVCaptureDeviceInput(device: videoCaptureDevice))
// Get Audio Device
let audioInput = AVCaptureDevice.defaultDevice(withMediaType: AVMediaTypeAudio)
//Add Audio Input
try self.captureSession.addInput(AVCaptureDeviceInput(device: audioInput))
self.previewLayer = AVCaptureVideoPreviewLayer(session: self.captureSession)
previewLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
previewLayer.connection.videoOrientation = AVCaptureVideoOrientation.portrait
self.videoView.layer.addSublayer(self.previewLayer)
//Add File Output
self.captureSession.addOutput(self.movieOutput)
captureSession.startRunning()
} catch {
print(error)
}
}
}
For more details refer this link:
https://medium.com/#santhosh3386/ios-avcapturesession-record-video-with-audio-23c8f8c9a8f8

Resources