I am doing a project on expression recognition in opencv and have successfully extracted the face region,i am having trouble building my own algorithm for the feature points extraction of a face,can someone help me with it?
since this is my first try to answer question, I will try to do my best. I cannot post more than two links, so I will try at least to give some hints.
Your question is quite broad. It depends on the type of application and requirements. Are you doing on-line detection, is it static etc.? Based on that, you should consider keypoint detection algorithm. I do not think it is a good idea to build your own algorithm, because OpenCV already has a lot of methods that you can choose from. All you have to do in most cases is to do some pre-processing, but it depends as well.
Most popular feature detection methods are: SURF (Opencv SURF), SIFT, ORB, FAST etc. Keep in mind that SURF and SIFT ar non-free. SURF and SIFT brings a lot of features and are quite accurate, somewhat scale and rotation invariant, but also a bit slow (especially on online tracking). FAST and ORB are fast, but they are more sensitive to noise and have their own disadvantages (see descriptions on OpenCV documentation). If I were you, I would try most of them and see which one does the best job (it is not difficult to test them).
Secondly, you have to choose descriptors. Very goog introduction is here:
Descriptors tutorial. There you will find all basic info. It is important to say, that you can mix various keypoint detection algorithms and feature description algorithms (but keep in mind that not all are compatible, tutorial will explain this).
I am out of links for this post, but OpenCV documentation provides a lot of sample code for this problem, so go ahead and have a look.
Hope you succeed. Good luck.
Related
This question is for those who have tried feature detection/matching methods on brain images - it is a broad one, and perhaps a bad one:
How could you tell if the method you used was "good enough?"
What does a successful matching/detection test look like for your data?
EDIT:
As of now, I am not trying to detect any distinct features in particular.
I'm using OpenCV's ORB, SIFT, SURF, etc detection methods, and seeing what they identify for features.
Sometimes, however, the orientation of the brain changes entirely from a
few set of images to the next set, so if I compare two images from these sets,the detection methods won't yield any effective
results (i.e. the matching will be distinctly, completely off). But if I compare images that look similar, but not identical,
the detection seems to work alright. Point is, it seems like detection works for frames that were taken around the same
time, but not over a long interval. I wonder if others have come across this and if they have found that detection methods
are still useful despite the fact.
First of all, you should specify what kind of features or for which purpose, the experiment is going to be performed.
Feature extraction is highly subjective in nature, it all depends on what type of problem you are trying to handle. There is no generic feature extraction scheme which works in all cases.
For example if the features are pointing out to some tumor classification or lesion, then of course there are different softwares you can use to extract and define your features.
There are different methods to detect the relevant features regarding to the application:
SURF algorithm (Speeded Up Robust Features)
PLOFS: It is a fast wrapper approach with a subset evaluation.
ICA or 'PCA
This paper is a very great review about brain MRI data feature extraction for tissue classification:
https://pdfs.semanticscholar.org/fabf/a96897dcb59ad9f04b5ff92bd15e1bd159ef.pdf
I found this paper very good o understand the difference between feature extraction techniques.
https://www.sciencedirect.com/science/article/pii/S1877050918301297
SURF is patented, as is SIFT. ORB and BRIEF are not patented, but their features are not scale-invariant, seriously limiting their usefulness in complex scenarios.
Are there any feature extractors that can extract scale-invariant features as fast as SURF and are not so strictly patented as SURF and SIFT?
Although you already choose BRISK, you might find FREAK interesting. Author claims to have better results than BRISK and ORB. I should also add that ORB is scale-invariant but has some problems in that area. So I would still recommend it for someone to try it.
The FREAK source code is compatible with OpenCV (how easy it is to merge them I do not know) and the author is working on submitting it to the OpenCV project.
EDIT:
FREAK is now part of opencv feature detectors / descriptors.
You can read here more about the differences between several feature detectors/extractors, and also a series of benchmarks which includes FREAK and other popular ones.
FREAK is supposed to be the fastest scale and rotation invariant descriptor extractor, it is open source and you can use it easily as it is implemented in OpenCV. You need a binary matcher that uses the Hamming Distance, like the BruteForceMatcher.
Here you have an example on how to use it, easily replaceble by SIFT descriptor.
I ended up using Brisk, which is a feature detector with performance comparable to SURF but under the BSD licence. Plus, it's got a very nice open source C++ implementation that plugs in easily to the OpenCV FeatureDetector framework, so it's like 2 lines choosing to use Brisk over SURF in your code.
You might try multi-scale histogram of oriented gradients. It won't be fully scale-invariant, but if your data are constrained with a reasonable set of scale limits (often the case in practice) then this can probably work for you.
Another approach, depending totally on what your intended application is, would be to leverage poselets, even if they are built on top of a non-scale-invariant descriptor like plain histogram of oriented gradient, or appearance models. If the annotations in your training data include examples of different items for detection all at different scales, then the Procrustes-style distance used in Poselets for training should take care of a lot of scale-invariance. This may not be satisfactory though if your primary application is not localized detection of parts.
As an aside, I think it's rather unfortunate that SIFT and SURF were capable of being patented this way, given that they were (at least in part) funded with tax-payer dollars through grants.
You can try KAZE, it is supposed to be faster and more precise (the videos seem fancy, but I did not get to try it myself yet). There is also an accelerated version of it available.
Pablo F. Alcantarilla, Adrien Bartoli and Andrew J. Davison, "KAZE Features", In European Conference on Computer Vision (ECCV). Fiorenze*, Italy. October 2012.
Pablo F. Alcantarilla, Jesús Nuevo and Adrien Bartoli, "Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces", In British Machine Vision Conference (BMVC). Bristol, UK. September 2013
The source codes, videos and more can be found at the author's website.
*Firenze
FREAK is now included in OpenCV 2.4.2, you can use it easily :-)
Detection of landmark points and finding hog features(it's not patented ) by forming patches around landmark points .
I know that there are many detection techniques in OpenCV, such as SURF, STAR, ORB etc...but those techniques are for feature detection of new video feed, not for dealing with specific instances of objects that require prior learning. OpenCV's documentation isn't quite as easy to flip through and I've yet been able to find anything besides Haar, which I know deals best with face recognition.
So are there any other techniques besides Haar? The Haar technique dates back to research 10 years ago, so ideally I hope that there have been some more advances since then that have been implemented in OpenCV.
If you are looking for OpenCV machine learning type algorithms, check out this link.
For a state of the art on-the-fly object detection algorithm, have a look at OpenTLD. It uses bounding boxes and random forests to learn about an object over time. Check out the demo video here.
Also check out the matching_to_many_images.cpp sample from OpenCV. It uses feature descriptors to match objects much like Google Goggles works. A related example to this is the bagofwords_classification.cpp sample. It may be what you are looking for in this case. It uses feature detectors (SURF, SIFT, etc...) to detect objects and then classify them by comparing the relative positions of the features to a learned database of features. Have a look also at this tutorial from MIT.
The latentsvmdetect.cpp may also be a good starting point for you.
Hope that helps!
In brief, what are the available options for implementing the Tracking of a particular Image(A photo/graphic/logo) in webcam feed using OpenCv?In particular i am trying to collate opinion about the following:
Would HaarTraining be overkill(considering that it is not 3d objects but simply Images to be tracked) or is it the only way out?
Have tried Template Matching, Color-based detection but these don't offer reliable tracking under varying illumination/Scale/Orientation at all.
Would SIFT,SURF feature matching work as reliably in video as with static image
comparison?
Am a relative beginner to OpenCV , as is evident by my previous queries on SO (very helpful replies). Any cues or links to what could be good resources for beginning NFT implementation with OpenCV?
Can you talk a bit more about your requirements? Namely, what type of appearance variations do you expect/how much control you have over the environment. What type of constraints do you have in terms of speed/power/resource footprint?
Without those, I can only give some general assessment to the 3 paths you are talking about.
1.
Haar would work well and fast, particularly for instance recognition.
Note that Haar doesn't work all that well for 3D unless you train with a full spectrum of templates to cover various perspectives. The poster child application of Haar cascades is Viola Jones' face detection system which is largely geared towards frontal faces (can certainly be trained for many other things)
For a tutorial on doing Haar training using OpenCV, see here.
2.
Try NCC or better yet, Lucas Kanade tracking (cvCalcOpticalFlowPyrLK which is a pyramidal as in coarse-to-fine LK - a 4 level pyramid usually works well) for a template. Usually good upto 10% scale or 10 degrees rotation without template changes. Beyond that, you can have automatically evolving templates which can drift over time.
For a quick Optical Flow/tracking tutorial, see this.
3.
SIFT/SURF would indeed work very well. I'd suggest some additional geometric verification step to remove spurious matches.
I'd be a bit concerned about the amount of computational time involved. If there isn't significant illumination/scale/in-plane rotation, then SIFT is probably overkill. If you truly need it, check out Changchang Wu's excellent SIFTGPU implmentation. Note: 3rd party, not OpenCV.
It seems that none of the methods when applied alone could bring reliable results unless it is a hobby project. Probably some adaptive algorithm would be more or less acceptable. For example see a famous opensource project where they use machine learning.
I was trying to build a basic Face Recognition system (PCA-Eigenfaces) using OpenCV 2.2 (from Willow Garage). I understand from many of the previous posts on Face Recognition that there is no standard open source library that can provide all the face recognition for you.
Instead, I would like to know if someone has used the functions(and integrated them):
icvCalcCovarMatrixEx_8u32fR
icvCalcEigenObjects_8u32fR
icvEigenProjection_8u32fR
et.al in the eigenobjects.cpp to form a face recognition system, because the functions seem to provide much of the required functionality along with cvSvd?
I am having a tough time trying to understand to do so since I am new to OpenCV.
Update: OpenCV 2.4.2 now comes with the very new cv::FaceRecognizer. Please see the very detailed documentation at:
http://docs.opencv.org/trunk/tutorial_face_main.html
I worked on a project with CV to recognize facial features. Most people don't understand the difference between Biometrics and Facial Recognition. There is a huge difference based on the fact that Biometrics is mainly based on histogram density matching while Facial Recognition implements this and vector support based on feature recognition from density. Check out the following link. This is the library you want to use if you are pursuing CV and Facial Recognition: www.betaface.com . Oleksander is awesome and based out of Germany, but he answers questions which is nice.
With OpenCV it's easy to get started with face detection. It comes with some predefined sets for feature detection, including face detection.
You might already know this one: OpenCV Wiki, FaceDetection
The important functions in this example are cvLoad and cvHaarDetectObjects. The first one loads the classifier and the second one applies it to an image.
The standard classifiers work pretty well. Of course you can train your own classifiers, if the standard ones don't fit your purpose.
As you said there are a lot of algorithms for face detection. Some of them might provide better results, but OpenCV is definitively a good start.