Is there any way in which we can limit the number of keypoints to the 100 in OPENCV SURF?
Will the keypoints obtained be ordered according to their strength?
How to obtain the strength of the descriptor?
I am working on OPENCV in a LINUX system with a cpp program.
regards,
shiksha
My code is:
int main( int argc, char** argv )
{
Mat img_1 = imread( argv[1], CV_LOAD_IMAGE_GRAYSCALE );
Mat img_2 = imread( argv[2], CV_LOAD_IMAGE_GRAYSCALE );
//-- Step 1: Detect the keypoints using SURF Detector
int minHessian = 500;
SurfFeatureDetector detector( minHessian,1,2,false,true );
std::vector<KeyPoint> keypoints_1p;
std::vector<KeyPoint> keypoints_2p;
detector.detect( img_1, keypoints_1p );
detector.detect( img_2, keypoints_2p);
// computing descriptors
SurfDescriptorExtractor extractor(minHessian,1,1,1,0);
Mat descriptors1, descriptors2;
extractor.compute(img_1, keypoints_1p, descriptors1);
extractor.compute(img_2, keypoints_2p, descriptors2);
You can get at most 100. I could imagine images (say for example a constant image) that have no SIFT descriptor. There are many ways to limit the keypoints to 100. There are easy solutions and hard solutions to your problem. You can get at most 100, by randomly selecting 100 keypoints from as many keypoints you get.
There is no such thing as the strength of the keypoint. You're going to have to define your own concept of strength.
There are a wide variety of parameter in the original Lowe paper that filter the keypoints (one of them is that they don't match an image edge, section 4.1 of Lowe's paper). There are 2 or 3 other parameters. You would need to adjust the parameters systematically in such a way that you only get 100. If you get less than 100 you filter less, and if you get more than 100 you filter more.
see the question here. And see my answer there how to limit the number of keypoints.
Related
Greetings for the past week (or more) I've been struggling with a problem.
Scenario:
I am developing an app which will allow an expert to create a recipe using a provided image of something to be used as a base. The recipe consists of areas of interests. The program's purpose is to allow non experts to use it, providing images similar to that original and the software cross checks these different areas of interest from the Recipe image to the Provided image.
One use-case scenario could be banknotes. The expert would select an area on an a good picture of a banknote that is genuine, and then the user would provide the software with images of banknotes that need to be checked. So illumination, as well as capturing device could be different.
I don't want you guys to delve into the nature of comparing banknotes, that's another monster to tackle and I got it covered for the most part.
My Problem:
Initially I shrink one of the two pictures to the size of the smaller one.
So now we are dealing with pictures having the same size. (I actually perform the shrinking to the areas of interest and not the whole picture, but that shouldn't matter.)
I have tried and used different methodologies compare these parts but each one had it's limitations due to the nature of the images. Illumination might be different, provided image might have some sort of contamination etc.
What have I tried:
Simple image similarity comparison using RGB difference.
Problem is provided image could be totally different but colours could be similar. So I would get high percentages on "totally" different banknotes.
SSIM on RGB Images.
Would give really low percentage of similarity on all channels.
SSIM after using sobel filter.
Again low percentage of similarity.
I used SSIM from both Scikit in python and SSIM from OpenCV
Feature matching with Flann.
Couldn't find a good way to use detected matches to extract a similarity.
Basically I am guessing that I need to use various methods and algorithms to achieve the best result. My gut tells me that I will need to combine RGB comparison results with a methodology that will:
Perform some form of edge detection like sobel.
Compare the results based on shape matching or something similar.
I am an image analysis newbie and I also tried to find a way to compare, the sobel products of the provided images, using mean and std calculations from openCV, however I either did it wrong, or the results I got were useless anyway. I calculated the eucledian distance between the vectors that resulted from mean and std calculation, however I could not use the results mainly because I couldn't see how they related between images.
I am not providing code I used, firslty because I scrapped some of it, and secondly because I am not looking for a code solution but a methodology or some direction to study-material. (I've read shitload of papers already).
Finally I am not trying to detect similar images, but given two images, extract the similarity between them, trying to bypass small differences created by illumination or paper distortion etc.
Finally I would like to say that I tested all the methods by providing the same image twice and I would get 100% similarity, so I didn't totally fuck it up.
Is what I am trying even possible without some sort of training sets to teach the software what are the acceptable variants of the image? (Again I have no idea if that even makes sense :D )
I think you can try Feature Matching, like SURF alogrithm, FLANN
https://docs.opencv.org/3.3.0/dc/dc3/tutorial_py_matcher.html
http://www.coldvision.io/2016/06/27/object-detection-surf-knn-flann-opencv-3-x-cuda/
Example of Feature Detection using SURF : https://docs.opencv.org/3.0-beta/doc/tutorials/features2d/feature_detection/feature_detection.html
#include <stdio.h>
#include <iostream>
#include "opencv2/core.hpp"
#include "opencv2/features2d.hpp"
#include "opencv2/xfeatures2d.hpp"
#include "opencv2/highgui.hpp"
using namespace cv;
using namespace cv::xfeatures2d;
void readme();
/** #function main */
int main( int argc, char** argv )
{
if( argc != 3 )
{ readme(); return -1; }
Mat img_1 = imread( argv[1], IMREAD_GRAYSCALE );
Mat img_2 = imread( argv[2], IMREAD_GRAYSCALE );
if( !img_1.data || !img_2.data )
{ std::cout<< " --(!) Error reading images " << std::endl; return -1; }
//-- Step 1: Detect the keypoints using SURF Detector
int minHessian = 400;
Ptr<SURF> detector = SURF::create( minHessian );
std::vector<KeyPoint> keypoints_1, keypoints_2;
detector->detect( img_1, keypoints_1 );
detector->detect( img_2, keypoints_2 );
//-- Draw keypoints
Mat img_keypoints_1; Mat img_keypoints_2;
drawKeypoints( img_1, keypoints_1, img_keypoints_1, Scalar::all(-1), DrawMatchesFlags::DEFAULT );
drawKeypoints( img_2, keypoints_2, img_keypoints_2, Scalar::all(-1), DrawMatchesFlags::DEFAULT );
//-- Show detected (drawn) keypoints
imshow("Keypoints 1", img_keypoints_1 );
imshow("Keypoints 2", img_keypoints_2 );
waitKey(0);
return 0;
}
/** #function readme */
void readme()
{ std::cout << " Usage: ./SURF_detector <img1> <img2>" << std::endl; }
Ok after some digging around, this is what I came with :
#!/usr/bin/env
import numpy as np
import cv2
import sys
import matplotlib.image as mpimg
from skimage import io
from skimage import measure
import time
s = 0
imgA = cv2.imread(sys.argv[1])
imgB = cv2.imread(sys.argv[2])
#imgA = cv2.imread('imageA.bmp')
#imgB = cv2.imread('imageB.bmp')
imgA = cv2.cvtColor(imgA, cv2.COLOR_BGR2GRAY)
imgB = cv2.cvtColor(imgB, cv2.COLOR_BGR2GRAY)
ret,imgA = cv2.threshold(imgA,127,255,0)
ret,imgB = cv2.threshold(imgB,127,255,0)
imgAContours, contoursA, hierarchyA = cv2.findContours(imgA, cv2.RETR_TREE , cv2.CHAIN_APPROX_NONE)
imgBContours, contoursB, hierarchyB = cv2.findContours(imgB, cv2.RETR_TREE , cv2.CHAIN_APPROX_NONE)
imgAContours = cv2.drawContours(imgAContours,contoursA,-1,(0,0,0),1)
imgBContours = cv2.drawContours(imgBContours,contoursB,-1,(0,0,0),1)
imgAContours = cv2.medianBlur(imgAContours,5)
imgBContours = cv2.medianBlur(imgBContours,5)
#s = 100 * 1/(1+cv2.matchShapes(imgAContours,imgBContours,cv2.CONTOURS_MATCH_I2,0.0))
#s = measure.compare_ssim(imgAContours,imgBContours)
#equality = np.equal(imgAContours,imgBContours)
total = 0.0
sum = 0.0
for x in range(len(imgAContours)):
for y in range(len(imgAContours[x])):
total +=1
t = imgAContours[x,y] == imgBContours[x,y]
if t:
sum+=1
s = (sum/total) * 100
print(s)
Basically I preprocess the two images as simply as possible, then I find the contours. Now the matchShapes function from openCV was not giving me the results I wanted.
So I create two images using the information from the contours, and then I apply a median blur filter.
Currently, I am doing a simply boolean check pixel to pixel. However I am planning to change this in the future, making it smarter. Probably with some array math.
If anyone has any suggestions, they are welcome.
I have an opencv application in which I need to store a 2d point's coordinates, and the feature descriptor for that point.
For example, after i run the following code:
std::vector<KeyPoint> keypoints_1;
Mat descriptors_1;
Ptr<FeatureDetector> detector = ORB::create();
Ptr<DescriptorExtractor> descriptor = ORB::create();
detector->detect ( img_1,keypoints_1 );
descriptor->compute ( img_1, keypoints_1, descriptors_1 );
I get descriptors_1, which is every descriptor for every Keypoint'
How can I get the descriptor that corresponds with, for example, keypoints_1[12], and store it as a Mat?
Neither descriptors_1.rows, nor descriptors_1.cols has the same size as the number of keypoints.
descriptor->compute may remove key points from the array - if it can't compute the descriptor for a given key point. After calling compute the two matrices should have the same number of rows.
Whats the difference between SiftFeatureDetector() and Ptr. They both apparently have the same function. The opencv tutorial uses SiftFeatureDetector but when clicking on the official documentation they use Ptr and have no mention of SiftFeatureDetector(), so I cant read up on it. as in the tutorial they used this: int minHessian = 400; SurfFeatureDetector detector( minHessian ); and I dont know what the minHessian is supposed to do.
Also I tried them both on the same image and they both have the same result, then why are they different?
int _tmain(int argc, _TCHAR* argv[])
{
//initModule_nonfree();
Mat img;
img = imread("c:\\box.png", 0);
//cvtColor( img, gry, CV_BGR2GRAY );
//SiftFeatureDetector detector;
//vector<KeyPoint> keypoints;
//detector.detect(img, keypoints);
Ptr<FeatureDetector> feature_detector = FeatureDetector::create("SIFT");
vector<KeyPoint> keypoints;
feature_detector->detect(img, keypoints);
Mat output;
drawKeypoints(img, keypoints, output, Scalar::all(-1));
namedWindow("meh", CV_WINDOW_AUTOSIZE);
imshow("meh", output);
waitKey(0);
return 0;
}
Thank you
EDIT: See the correction by #gantzer89 in the comments below. (Leaving my original text in place for historical clarity.)
In my general experience, using the FeatureDetector::create() syntax (discussed here in the "official documentation" you cited) allows the flexibility to specify your algorithm at runtime via a parameter file, while the more specific classes, such as SiftFeatureDetector, provide more opportunities for customization.
The create() methods start with a set of default algorith-specific parameters, while the algorithim-specific classes allow customization of these parameters upon construction. Thus, the create() method is assigning a default value to minHessian, while the SiftFeatureDetector constructor provides the opportunity to choose a value of minHessian.
As a rule of thumb, if you want to quickly experiment with which algorithm to use, use the create() syntax, and if you want to experiment with fine-tuning a particular algorithm, use the algorithm-specific class constructor.
The Brisk detector shown below gives me no keypoints. Can somebody please suggest a problem.
I will try to explain what I am doing below with some of the codes.
#include "opencv2/features2d/features2d.hpp"
using namespace cv;
using namespace std;
Ptr<FeatureDetector> detector;
detector = FeatureDetector::create("BRISK");
// the filename is given some path
Mat img = imread(filename, 0);
CV_Assert( !img.empty() );
vector<KeyPoint> kp;
detector->detect(img, kp);
So, when I debug this and check the keypoint (kp) it says <0 items>
with similar code other detector like ORB, SIFT or SURF work as intended!
Can somebody please suggest a solution.
I am using opencv 2.4.9 with Qt creator 2.5.2
Thanks
Okay, I got it myself!
For someone interested, the default values of the parameters for BriskFeatureDetector i.e octaves = 3 and thres = 30 doesn't gave me keypoints at all. But when I change the octaves = 0 as shown in the original author's demo to use AGAST detector of Brisk, it gave me a considerable amount of keypoints.
Thanks and enjoy!
I have an image and i want to locate key points by using SIFT detector and group them, then i want to generate local features for each key point by using SIFT, would you please help me how I can do it ? Please give me any suggestions
I really appreciate your help
I'm not sure that I understand what you mean, but if you extract SIFT features from an image, you automatically get the feature descriptor which is used to compare features to each other. Of course you also get the feature location, size, direction and hessian value with it.
While you can group those features by there position in the image, but there is currently no way that I'm aware of to compare those groups, since they may be locally related, but can have wildly different feature descriptors.
Also I would suggest SURF. It is faster and not patent encumbered.
Have a look at the examples from OpenCV if you want specific instructions on how to retrieve and compare descriptors.
If you are using opencv here are the commands to do it, else if you are using the matlab see the link MATCHING_using surf
USING OPENCV::
// you can change the parameters for your requirement
double hessianThreshold=200;
int octaves=3;
int octaveLayers=4;
bool upright=false;
vector<KeyPoint>keypoints;
//The detector detects the keypoints in an image here image is RGBIMAGE of Mat type
SurfFeatureDetector detector( hessianThreshold, octaves, octaveLayers, upright );
detector.detect(RGB_IMAGE, keypoints);
//The extractor computesthe local features around the keypoints
SurfDescriptorExtractor extractor;
Mat descriptors;
extractor.compute( last_ref, keypoints, descriptors);
// all the key points local features are stored in rows one after another in descriptors matrix...
Hope it is useful:)