Calculate similarity score between scene and template object - ios

Question

How can I calculate some comparable similarity score which tells me how similar the img_scene is compared to img_object.
When I render the img_matches, the homography successfully renders the boundaries of the found object in the scene, but I need some comparable score like if (score > THRESHOLD) { /* have match */ } else { /* dont have match */ }.
Mat img_scene = srcImage;
Mat img_object = _templateImage;
//-- Step 1: Detect the keypoints using SURF Detector
SurfFeatureDetector detector(_minHessian);
std::vector<KeyPoint> keypoints_object, keypoints_scene;
detector.detect(img_object, keypoints_object);
detector.detect(img_scene, keypoints_scene);
//-- Step 2: Calculate descriptors (feature vectors)
SurfDescriptorExtractor extractor;
Mat descriptors_object, descriptors_scene;
extractor.compute(img_object, keypoints_object, descriptors_object);
extractor.compute(img_scene, keypoints_scene, descriptors_scene);
if (descriptors_object.type() != descriptors_scene.type())
return;
//-- Step 3: Matching descriptor vectors using FLANN matcher
FlannBasedMatcher matcher;
std::vector<DMatch> matches;
matcher.match(descriptors_object, descriptors_scene, matches);
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for (size_t i = 0; i < (size_t)descriptors_object.rows; i++ ) {
double dist = matches[i].distance;
if (dist < min_dist) min_dist = dist;
if (dist > max_dist) max_dist = dist;
}
//-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist )
std::vector<DMatch> good_matches;
for(size_t i = 0; i < (size_t)descriptors_object.rows; i++) {
if (matches[i].distance < 2 * min_dist) {
good_matches.push_back(matches[i]);
}
}
if (good_matches.size() < 4)
return;
Mat img_matches;
drawMatches(img_object, keypoints_object, img_scene, keypoints_scene,
good_matches, img_matches, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);
//-- Localize the object
std::vector<Point2f> obj;
std::vector<Point2f> scene;
for (size_t i = 0; i < (size_t)good_matches.size(); i++) {
//-- Get the keypoints from the good matches
obj.push_back(keypoints_object[(size_t)good_matches[i].queryIdx].pt);
scene.push_back(keypoints_scene[(size_t)good_matches[i].trainIdx].pt);
}
vector<uchar> mask;
Mat H = findHomography(obj, scene, CV_RANSAC, 3, mask);
//-- Get the corners from the image_1 (the object to be "detected")
std::vector<Point2f> obj_corners(4);
obj_corners[0] = cvPoint(0, 0);
obj_corners[1] = cvPoint(img_object.cols, 0);
obj_corners[2] = cvPoint(img_object.cols, img_object.rows);
obj_corners[3] = cvPoint(0, img_object.rows);
std::vector<Point2f> scene_corners(4);
perspectiveTransform(obj_corners, scene_corners, H);
//-- Draw lines between the corners (the mapped object in the scene - image_2 )
line(img_matches, scene_corners[0] + Point2f(img_object.cols, 0), scene_corners[1] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[1] + Point2f(img_object.cols, 0), scene_corners[2] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[2] + Point2f(img_object.cols, 0), scene_corners[3] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[3] + Point2f(img_object.cols, 0), scene_corners[0] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
UPDATE:
Here is the working solution as #mikesapi proposed:
...
//-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist )
std::vector<DMatch> good_matches;
double good_matches_sum = 0.0;
for (size_t i = 0; i < matches.size(); i++ ) {
if( matches[i].distance < max(2*min_dist, 0.02) ) {
good_matches.push_back(matches[i]);
good_matches_sum += matches[i].distance;
}
}
double score = (double)good_matches_sum / (double)good_matches.size();
if (score < 0.18) {
// have match
} else {
// dont have match
}
...

Answer 1

A similarity score is greater if the object and scene are more similar (as opposed to a dissimilarity score, where a higher score means they are more dissimilar). Since you are using distances with FLANN (which I assume is giving you approximate euclidean distances between descriptors) a dissimilarity score is easier to generate, since euclidean distance is greater if descriptors are further apart in the descriptor space, and small if they are close together.
One simple way to generate a dissimilarity score would be to:
1. For each descriptor in the object image: calculate the minimum distance to each descriptor in the scene image.
2. Sum the (minimum) distances, and normalize by the number of descriptors in the object image.
Then you will have a single score quantifying the match between the object and the scene.