on device face recognition - Swift [closed] - ios

Question

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have an app that takes all the images of the user (all the Assets from the Photo app). After that the app should run throw all the images and detect faces and return their facial landmarks, and then look in the database to see if there is any friend with the same landmark (recognizing friend faces), similar to what Facebook do on moments app and on the web. The app will then show all the photos that the friend appear in them. Important part of my app is the user privacy, so I would like to keep the entire process on the device, and not sending it to online service. Another benefit of keeping it on the device is that every user in my app can have thousands of images, and working with external service will be expansive and might low down the performance (if every image need to be sent to the server).
From the research that I done there are many online services (but they don't fit my requirements - keeping the process offline). There is also the CIDector that detect the faces, and then you can return few features such as eye location and mouth location (which I don't believe that is good enough for reliable recognition). I also heard about Luxand, openCV, and openFace, which are all on device recognition, but are C++ class, which make it difficult to integrate with swift project (the documentation are not very good, and don't explain how do integrate it to your project and how to perform face recognition on swift).
So my question is if there is any way to perform face detection that return the facial landmarks on the device?
If not is there any other way or service I could use.
Also if there is any efficient and fast way to perform face detection and recognition, if a user could have thousand of images.
By the way, I am in early stage of development and I am looking for free services that I could use for the development stage.

Answer 1

iOS have native face detection in CoreImage framework that works pretty cool. You can as well detect eyes etc. Just check out this code, there's show how you can work with it.
func detect() {
guard let personciImage = CIImage(image: personPic.image!) else {
return
}
let accuracy = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: accuracy)
let faces = faceDetector.featuresInImage(personciImage)
// converting to other coordinate system
let ciImageSize = personciImage.extent.size
var transform = CGAffineTransformMakeScale(1, -1)
transform = CGAffineTransformTranslate(transform, 0, -ciImageSize.height)
for face in faces as! [CIFaceFeature] {
print("Found bounds are \(face.bounds)")
// calculating place for faceBox
var faceViewBounds = CGRectApplyAffineTransform(face.bounds, transform)
let viewSize = personPic.bounds.size
let scale = min(viewSize.width / ciImageSize.width,
viewSize.height / ciImageSize.height)
let offsetX = (viewSize.width - ciImageSize.width * scale) / 2
let offsetY = (viewSize.height - ciImageSize.height * scale) / 2
faceViewBounds = CGRectApplyAffineTransform(faceViewBounds, CGAffineTransformMakeScale(scale, scale))
faceViewBounds.origin.x += offsetX
faceViewBounds.origin.y += offsetY
let faceBox = UIView(frame: faceViewBounds)
faceBox.layer.borderWidth = 3
faceBox.layer.borderColor = UIColor.redColor().CGColor
faceBox.backgroundColor = UIColor.clearColor()
personPic.addSubview(faceBox)
if face.hasLeftEyePosition {
print("Left eye bounds are \(face.leftEyePosition)")
}
if face.hasRightEyePosition {
print("Right eye bounds are \(face.rightEyePosition)")
}
}
}