Swift - Realtime images from cam to server - ios

Question

I hope someone can help me!
I am making an app that sends frames of the camera to server, and server make some process. App sends 5-8 images per second (On NSData format)
I have tried different ways to do that, the two methods works but have different problems.
I will explain those situations, and maybe someone can help me.
First situation i tried is using AVCaptureVideoDataOutput mode.
Code below:
let captureSession = AVCaptureSession()
captureSession.sessionPreset=AVCaptureSessionPresetiFrame960x540
captureSession.addInput(AVCaptureDeviceInput(device: captureDevice, error: &error))
let output=AVCaptureVideoDataOutput();
output.videoSettings=[kCVPixelBufferPixelFormatTypeKey:kCVPixelFormatType_32BGRA]
let cameraQueue = dispatch_queue_create("cameraQueue", DISPATCH_QUEUE_SERIAL)
output.setSampleBufferDelegate(self, queue: cameraQueue)
captureSession.addOutput(output)
videoPreviewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
videoPreviewLayer?.videoGravity = AVLayerVideoGravityResizeAspectFill
videoPreviewLayer?.frame = view.layer.bounds
viewPreview?.layer.addSublayer(videoPreviewLayer)
captureSession.startRunning()
This view delegates:
AVCaptureMetadataOutputObjectsDelegate
AVCaptureVideoDataOutputSampleBufferDelegate
and call the delegate method:
func captureOutput(captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBufferRef!, fromConnection connection: AVCaptureConnection!)
{
let imagen:UIImage=imageFromSampleBuffer(sampleBuffer)
let dataImg:NSdata=UIImageJPEGRepresentation(imagen,1.0)
//Here I send the NSData to server correctly.
}
This method call imageFromSampleBuffer and converts samplebuffer to uiimage.
func imageFromSampleBuffer(sampleBuffer :CMSampleBufferRef) -> UIImage {
let imageBuffer: CVImageBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)
CVPixelBufferLockBaseAddress(imageBuffer, 0)
let baseAddress: UnsafeMutablePointer<Void> = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, Int(0))
let bytesPerRow: Int = CVPixelBufferGetBytesPerRow(imageBuffer)
let width: Int = CVPixelBufferGetWidth(imageBuffer)
let height: Int = CVPixelBufferGetHeight(imageBuffer)
let colorSpace: CGColorSpaceRef = CGColorSpaceCreateDeviceRGB()
let bitsPerCompornent: Int = 8
var bitmapInfo = CGBitmapInfo((CGBitmapInfo.ByteOrder32Little.rawValue | CGImageAlphaInfo.PremultipliedFirst.rawValue) as UInt32)
let newContext: CGContextRef = CGBitmapContextCreate(baseAddress, width, height, bitsPerCompornent, bytesPerRow, colorSpace, bitmapInfo) as CGContextRef
let imageRef: CGImageRef = CGBitmapContextCreateImage(newContext)
let resultImage = UIImage(CGImage: imageRef, scale: 1.0, orientation: UIImageOrientation.Right)!
return resultImage
}
Here finish the first method to do that, the problem is "infinite memory use", and app crashed after....2 minutes.
I Debug and problem is on UIImageJPEGRepresentation(imagen,1.0) method, there are any form to release memory after use the method???
Second (and I think best way i found) to do that is using "AVCaptureStillImageOutput"
Code below:
var stillImageOutput: AVCaptureStillImageOutput = AVCaptureStillImageOutput()
if session.canAddOutput(stillImageOutput){
stillImageOutput.outputSettings = [AVVideoCodecKey: AVVideoCodecJPEG]
session.addOutput(stillImageOutput)
self.stillImageOutput = stillImageOutput
}
var timer = NSTimer.scheduledTimerWithTimeInterval(0.2, target: self, selector: Selector("methodToBeCalled"), userInfo: nil, repeats: true)
func methodToBeCalled(){
dispatch_async(self.sessionQueue!, {
// Update the orientation on the still image output video connection before capturing.
let videoOrientation = (self.previewView.layer as! AVCaptureVideoPreviewLayer).connection.videoOrientation
self.stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo).videoOrientation = videoOrientation
self.stillImageOutput!.captureStillImageAsynchronouslyFromConnection(self.stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo), completionHandler: {
(imageDataSampleBuffer: CMSampleBuffer!, error: NSError!) in
if error == nil {
let dataImg:NSdata= AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageDataSampleBuffer)
//Here I send the NSData to server correctly.
}else{println(error)}
})
})
}
This works perfectly and without memory leaks, but when the app takes a Screenshot, the phone makes the tipical sound of "take a photo", and i can not allow it, there are any way to do that without make the sound??.
If someone needs the code i can share the links where i found them.
Thanks a lot!

Answer 1

Did you ever manage to solve this problem yourself?
I stumbled upon this question because I am converting a Objective-C project with an AVCaptureSession to Swift. What my code does differently is discard late frames in the AVCaptureVideoDataOutput, perhaps this is causing your memory problem.
output.alwaysDiscardsLateVideoFrames = true
Insert this line right after you define the video data output and before you create the queue:
let output=AVCaptureVideoDataOutput();
output.videoSettings=[kCVPixelBufferPixelFormatTypeKey:kCVPixelFormatType_32BGRA]
output.alwaysDiscardsLateVideoFrames = true
let cameraQueue = dispatch_queue_create("cameraQueue", DISPATCH_QUEUE_SERIAL)
I am, of course, referring to the first of your two solutions.

Answer 2

There is no way to disallow users from taking a screenshot. Not even snapchat can do that.