How to define customized detector for iOS AVFoundation? - ios

Question

I want to detect some triangle patterns from camera input using iPhone. I found some example code that can detect QR/bar code using AVFoundation. The main part seems to be AVMetadataMachineReadableCodeObject class. Here is some sample code from AppCoda:
func captureOutput(captureOutput: AVCaptureOutput!, didOutputMetadataObjects metadataObjects: [AnyObject]!, fromConnection connection: AVCaptureConnection!) {
// Check if the metadataObjects array is not nil and it contains at least one object.
if metadataObjects == nil || metadataObjects.count == 0 {
qrCodeFrameView?.frame = CGRectZero
messageLabel.text = "No barcode/QR code is detected"
return
}
// Get the metadata object.
let metadataObj = metadataObjects[0] as! AVMetadataMachineReadableCodeObject
// Here we use filter method to check if the type of metadataObj is supported
// Instead of hardcoding the AVMetadataObjectTypeQRCode, we check if the type
// can be found in the array of supported bar codes.
if supportedBarCodes.contains(metadataObj.type) {
// if metadataObj.type == AVMetadataObjectTypeQRCode {
// If the found metadata is equal to the QR code metadata then update the status label's text and set the bounds
let barCodeObject = videoPreviewLayer?.transformedMetadataObjectForMetadataObject(metadataObj)
qrCodeFrameView?.frame = barCodeObject!.bounds
if metadataObj.stringValue != nil {
messageLabel.text = metadataObj.stringValue
}
}
In the above code, once a QR code is detected, a boundary box will be drawn and the text field will be updated. Similarly, AVMetadataFaceObject class is used in face detection applications. I saw from the reference that both classes are subclasses of AVMetadataObject.
I'm wondering if it is possible to customize a triangles detector by writing a subclass of AVMetadataObject, say, we call the subclass AVMetadataTriangleObject. (I have a readily available detection algorithm and have code written in Matlab. Transcribing it into swift shouldn't be tough.) If this approach is not possible, can anyone suggest alternative way(s) for achieving the above goal?
Thank you so much!

Answer 1

AVMetadataObject provides no hooks for implementing such a thing, and apart from AVAssetResourceLoader, AVFoundation does not allow much extension.
I think you should transcribe your algorithm to swift and run it against the captured CMSampleBuffer images that you get when you capture using from an AVCaptureVideoDataOutput.