My project requires face recognition based on the availability of limited features which mainly include eyes and the forehead region.
But I am unable to find any good resources to perform the same.
There are mostly resources dealing with face detection to face recognition using opencv but I have not been able to find any resource on how I can use eye detection and its feature extraction to perform face recognition with opencv.
Any guidance regarding this problem or suggestions of alternatives would be helpful.
Thank you
My idea was to use virtual mask application on basic full face image and then to make standard face_recognition / dlib's face encodings with large 68-points model for further dlib face comparation. It works. Not 100%, especially on millions-of-images data set but better then without such approach.
https://github.com/bohdansok/Face_Recognition
I aim to detect and track vehicles with opencv using recorded on-road scenarios in real-time. I realize most of the algorithms/papers use Haar-like-features or HoG as feature-descriptors, and in most literature about vehicle detection there is not much investigation in using point-of-interest based approaches.
I mean in OpenCV there are so many nice edge/corner -based detectors like FAST, ORB, BRISK, ....why don't use them in junction with a nice descriptor and do some matching/classification afterwards?
And when is it actually better to use such a detector/descriptor strategy for object detection compared with a traditional Haar-cascade or HoG/SVM approach? Is it for robustness or performance reasons?
Regards
Obi
SURF is patented, as is SIFT. ORB and BRIEF are not patented, but their features are not scale-invariant, seriously limiting their usefulness in complex scenarios.
Are there any feature extractors that can extract scale-invariant features as fast as SURF and are not so strictly patented as SURF and SIFT?
Although you already choose BRISK, you might find FREAK interesting. Author claims to have better results than BRISK and ORB. I should also add that ORB is scale-invariant but has some problems in that area. So I would still recommend it for someone to try it.
The FREAK source code is compatible with OpenCV (how easy it is to merge them I do not know) and the author is working on submitting it to the OpenCV project.
EDIT:
FREAK is now part of opencv feature detectors / descriptors.
You can read here more about the differences between several feature detectors/extractors, and also a series of benchmarks which includes FREAK and other popular ones.
FREAK is supposed to be the fastest scale and rotation invariant descriptor extractor, it is open source and you can use it easily as it is implemented in OpenCV. You need a binary matcher that uses the Hamming Distance, like the BruteForceMatcher.
Here you have an example on how to use it, easily replaceble by SIFT descriptor.
I ended up using Brisk, which is a feature detector with performance comparable to SURF but under the BSD licence. Plus, it's got a very nice open source C++ implementation that plugs in easily to the OpenCV FeatureDetector framework, so it's like 2 lines choosing to use Brisk over SURF in your code.
You might try multi-scale histogram of oriented gradients. It won't be fully scale-invariant, but if your data are constrained with a reasonable set of scale limits (often the case in practice) then this can probably work for you.
Another approach, depending totally on what your intended application is, would be to leverage poselets, even if they are built on top of a non-scale-invariant descriptor like plain histogram of oriented gradient, or appearance models. If the annotations in your training data include examples of different items for detection all at different scales, then the Procrustes-style distance used in Poselets for training should take care of a lot of scale-invariance. This may not be satisfactory though if your primary application is not localized detection of parts.
As an aside, I think it's rather unfortunate that SIFT and SURF were capable of being patented this way, given that they were (at least in part) funded with tax-payer dollars through grants.
You can try KAZE, it is supposed to be faster and more precise (the videos seem fancy, but I did not get to try it myself yet). There is also an accelerated version of it available.
Pablo F. Alcantarilla, Adrien Bartoli and Andrew J. Davison, "KAZE Features", In European Conference on Computer Vision (ECCV). Fiorenze*, Italy. October 2012.
Pablo F. Alcantarilla, Jesús Nuevo and Adrien Bartoli, "Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces", In British Machine Vision Conference (BMVC). Bristol, UK. September 2013
The source codes, videos and more can be found at the author's website.
*Firenze
FREAK is now included in OpenCV 2.4.2, you can use it easily :-)
Detection of landmark points and finding hog features(it's not patented ) by forming patches around landmark points .
I know that there are many detection techniques in OpenCV, such as SURF, STAR, ORB etc...but those techniques are for feature detection of new video feed, not for dealing with specific instances of objects that require prior learning. OpenCV's documentation isn't quite as easy to flip through and I've yet been able to find anything besides Haar, which I know deals best with face recognition.
So are there any other techniques besides Haar? The Haar technique dates back to research 10 years ago, so ideally I hope that there have been some more advances since then that have been implemented in OpenCV.
If you are looking for OpenCV machine learning type algorithms, check out this link.
For a state of the art on-the-fly object detection algorithm, have a look at OpenTLD. It uses bounding boxes and random forests to learn about an object over time. Check out the demo video here.
Also check out the matching_to_many_images.cpp sample from OpenCV. It uses feature descriptors to match objects much like Google Goggles works. A related example to this is the bagofwords_classification.cpp sample. It may be what you are looking for in this case. It uses feature detectors (SURF, SIFT, etc...) to detect objects and then classify them by comparing the relative positions of the features to a learned database of features. Have a look also at this tutorial from MIT.
The latentsvmdetect.cpp may also be a good starting point for you.
Hope that helps!
I was trying to build a basic Face Recognition system (PCA-Eigenfaces) using OpenCV 2.2 (from Willow Garage). I understand from many of the previous posts on Face Recognition that there is no standard open source library that can provide all the face recognition for you.
Instead, I would like to know if someone has used the functions(and integrated them):
icvCalcCovarMatrixEx_8u32fR
icvCalcEigenObjects_8u32fR
icvEigenProjection_8u32fR
et.al in the eigenobjects.cpp to form a face recognition system, because the functions seem to provide much of the required functionality along with cvSvd?
I am having a tough time trying to understand to do so since I am new to OpenCV.
Update: OpenCV 2.4.2 now comes with the very new cv::FaceRecognizer. Please see the very detailed documentation at:
http://docs.opencv.org/trunk/tutorial_face_main.html
I worked on a project with CV to recognize facial features. Most people don't understand the difference between Biometrics and Facial Recognition. There is a huge difference based on the fact that Biometrics is mainly based on histogram density matching while Facial Recognition implements this and vector support based on feature recognition from density. Check out the following link. This is the library you want to use if you are pursuing CV and Facial Recognition: www.betaface.com . Oleksander is awesome and based out of Germany, but he answers questions which is nice.
With OpenCV it's easy to get started with face detection. It comes with some predefined sets for feature detection, including face detection.
You might already know this one: OpenCV Wiki, FaceDetection
The important functions in this example are cvLoad and cvHaarDetectObjects. The first one loads the classifier and the second one applies it to an image.
The standard classifiers work pretty well. Of course you can train your own classifiers, if the standard ones don't fit your purpose.
As you said there are a lot of algorithms for face detection. Some of them might provide better results, but OpenCV is definitively a good start.