I saved ORB and SIFT descriptors in xml.
I tried use Mat descriptors = imread("descriptor.xml")
but it didn't work.
it's blank.
Let say that you have stored the SIFT descriptor in descriptor.xml. Use the following code to assign the saved descriptor back to a Mat variable.
Mat sift_descriptor;
FileStorage fs("descriptor.xml", FileStorage::READ);
fs["descriptor"] >> sift_descriptor;
fs.release();
You have to save and load Mats that are not images with cv::FileStorage.
Related
I have a keras network model that takes INDArray but I don't know how to convert a Mat to an INDArray. I know that this is the same question of How to convert Mat (opencv) to INDArray (DL4J)? but it did not help me. Is there some API that performs this task? Thanks
You can use a NativeImageLoader for the conversion.
import org.datavec.image.loader.NativeImageLoader;
(...)
Mat cvImage();
// Fill in your Mat with something
NativeImageLoader nil = new NativeImageLoader();
INDArray image = nil.asMatrix(cvImage).div;
Make sure you have the dependency for datavec in your pom.xml. What error do you get?
i am using the following code in the descriptor_extractor_matcher.cpp sample to compute the descriptors of img1 (Mat descriptors01), write it to my disk and load it back (Mat descriptors1). (same steps for the keypoints, but code is rather much the same ...)
Ptr<DescriptorExtractor> descriptorExtractor = DescriptorExtractor::create( argv[2] );
...
Mat descriptors01;
descriptorExtractor->compute( img1, keypoints1, descriptors01 ); // compute descriptors
FileStorage storage("test.yml", FileStorage::WRITE); //save it to disc
storage << "blub" << descriptors01;
storage.release();
Mat descriptors1;
FileStorage storage1("test.yml", FileStorage::READ); // load it again
storage1["blub"] >> descriptors1;
storage1.release();
The keypoints & descriptors for image 2 are computed and used without saving and loading.
I am using only the loaded data (keypoints & descriptors) for image 1 for the matching, so for the descriptors: descriptors1.
Now here is the thing: if I compare the cases
A) Using the code above for computing, storing and loading;
B) Using only loaded data (without computing and store it again)
for the matching I get different results, as you can see in the pictures for keypoints aswell as for the matching descriptors. I would have expect no differences... What am I missing here? Must I compare 2 images, and cannot compare an image to a stored set of keypoints and it's descriptors ?
Of course I'm using the same values for [detectorType] [descriptorType] [matcherType] [matcherFilterType] [image1] [image2] [ransacReprojThreshold], by the way ;)
Thanks alot!
UPDATE:
It seems the issue is depending on the descriptor. Working with loaded descriptors works for SIFT and SURF, but not for ORB and other. Images: Results with different descriptors for case A and B:
Try repeating A or B individually and see if the results are coming out to be the same. I suspect they won't and I say that because, #1 Your object of interest has poor texture and that would result in poor descriptors. #2 The viewpoint change between the two images is huge and which leads to the problem of non-repeatability even for the best of the descriptors like SIFT.
Now, comes the part of how to solve this repeatability issue, #1 use some threshold on the norm of the descriptor so that only very strong features are used for matching. #2 use the epipolar constraint along with RANSAC to filter out wrong matches. I am attaching two images to show how the filter hugely affects the correspondences.
Using SURF to find correspondence between the two images (two images in red-cyan colormap)
After filtering the images using RANSAC using epipolar constraint.
Feel free to comment and discuss further over this issue. :-)
I am trying to extract different point descriptors (SIFT, SURF, ORB, BRIEF,...) to build Bag of Visual words. The problem seems to be that I am using very small images : 12x60px.
Using a dense extractor I am able to get some keypoints, but then when extracting the descriptor no data is extracted.
Here is the code :
vector<KeyPoint> points;
Mat descriptor; // descriptor of the current image
Ptr<DescriptorExtractor> extractor = DescriptorExtractor::create("BRIEF");
Ptr<FeatureDetector> detector(new DenseFeatureDetector(1.f,1,0.1f,6,0,true,false));
image = imread(filename, 0);
roi = Mat(image,Rect(0,0,12,60));
detector->detect(roi,points);
extractor->compute(roi,points,descriptor);
cout << descriptor << endl;
The result is [] (with BRIEF and ORB) and SegFault (with SURF and SIFT).
Does anyone have a clue on how to densely extract point descriptors from small images on OpenCV ?
Thanks for your help.
Indeed, I finally managed to work my way to a solution. Thanks for the help.
I am now using an Orb detector with initalised parameters instead of a random one, e.g:
Ptr<DescriptorExtractor> extractor(new ORB(500, 1.2f, 8, orbSize, 0, 2, ORB::HARRIS_SCORE, orbSize));
I had to explore the documentation of OpenCV thoroughly before finding the answer to my problem : Orb documentation.
Also if people are using the dense point extractor they should be aware that after the descriptor computing process they may have less keypoints than produced by the keypoint extractor. The descriptor computing removes any keypoints for which it cannot get the data.
BRIEF and ORB use a 32x32 patch to get the descriptor. Since it doesn't fit your image, they remove those keypoints (to avoid returning keypoints without descriptor).
In the case of SURF and SIFT, they can use smaller patches, but it depends on the scale provided by the keypoint. In this case, I guess they have to use a bigger patch and the same as before happens. I don't know why you get a segfault, though; maybe the SIFT/SURF descriptor extractors don't check that keypoints are inside the image boundaries, as BRIEF/ORB ones do.
I have an image and i want to locate key points by using SIFT detector and group them, then i want to generate local features for each key point by using SIFT, would you please help me how I can do it ? Please give me any suggestions
I really appreciate your help
I'm not sure that I understand what you mean, but if you extract SIFT features from an image, you automatically get the feature descriptor which is used to compare features to each other. Of course you also get the feature location, size, direction and hessian value with it.
While you can group those features by there position in the image, but there is currently no way that I'm aware of to compare those groups, since they may be locally related, but can have wildly different feature descriptors.
Also I would suggest SURF. It is faster and not patent encumbered.
Have a look at the examples from OpenCV if you want specific instructions on how to retrieve and compare descriptors.
If you are using opencv here are the commands to do it, else if you are using the matlab see the link MATCHING_using surf
USING OPENCV::
// you can change the parameters for your requirement
double hessianThreshold=200;
int octaves=3;
int octaveLayers=4;
bool upright=false;
vector<KeyPoint>keypoints;
//The detector detects the keypoints in an image here image is RGBIMAGE of Mat type
SurfFeatureDetector detector( hessianThreshold, octaves, octaveLayers, upright );
detector.detect(RGB_IMAGE, keypoints);
//The extractor computesthe local features around the keypoints
SurfDescriptorExtractor extractor;
Mat descriptors;
extractor.compute( last_ref, keypoints, descriptors);
// all the key points local features are stored in rows one after another in descriptors matrix...
Hope it is useful:)
I'm working on a face recognition project in which we are using PCA to reduce feature vector size of an image. The trouble is, during training, I create the PCA object by incorporating all the training images. Now, during testing, I need the PCA object obtained earlier.
I cannot seem to figure out how to write the PCA object to a file, so that I can use it during testing. One alternative is that I write it's eigenvectors to the file. But it would be so much more convenient to write the object itself. Is there a way to do this?
As far as I know, there is no generic way of saving PCA objects to a file. You will need to save eigenvectors, eigenvalues and mean to a file, and then put them into a new PCA after loading. You have to remember to use a format that doesn't lose precision, especially for mean.
Here is some example code:
#include "opencv2/core/core.hpp"
#include <iostream>
...
cv::PCA pca1;
cv::PCA pca2;
cv::Mat eigenval,eigenvec,mean;
cv::Mat inputData;
cv::Mat outputData1,outputData2;
//input data has been populated with data to be used
pca1(inputData,Mat()/*dont have previously computed mean*/,
CV_PCA_DATA_AS_ROW /*depends of your data layout*/);//pca is computed
pca1.project(inputData,outputData1);
//here is how to extract matrices from pca
mean=pca1.mean.clone();
eigenval=pca1.eigenvalues.clone();
eigenvec=pca1.eigenvectors.clone();
//here You can save mean,eigenval and eigenvec matrices
//and here is how to use them to make another pca
pca2.eigenvalues=eigenval;
pca2.eigenvectors=eigenvec;
pca2.mean=mean;
pca2.project(inputData,outputData2);
cv::Mat diff;//here some proof that it works
cv::absdiff(outputData1,outputData2,diff);
std::cerr<<sum(diff)[0]<<std::endl; //assuming Youre using one channel data, there
//is data only in first cell of the returned scalar
// if zero was printed, both output data matrices are identical
You may try this.
void save(const string &file_name,cv::PCA pca_)
{
FileStorage fs(file_name,FileStorage::WRITE);
fs << "mean" << pca_.mean;
fs << "e_vectors" << pca_.eigenvectors;
fs << "e_values" << pca_.eigenvalues;
fs.release();
}
int load(const string &file_name,cv::PCA pca_)
{
FileStorage fs(file_name,FileStorage::READ);
fs["mean"] >> pca_.mean ;
fs["e_vectors"] >> pca_.eigenvectors ;
fs["e_values"] >> pca_.eigenvalues ;
fs.release();
}
Here is the source.