Say I have a very simple image or shape such as this stick man drawing:
I also have a library of other simple images which I want to compare the first image to and determine the closest match:
Notice that the two stick men are not completely identical but are reasonably similar.
I want to be able to compare the first image to each image in my library until a reasonably close match is found. If necessary, my image library could contain numerous variations of the same image in order to help decide which type of image I have. For example:
My question is whether this is something that OpenCV would be capable of? Has it been done before, and if so, can you point me in the direction of some examples? Many thanks for your help.
Edit: Through my searches I have found many examples of people who are comparing images, or even people that are comparing images which have been stretched or skewed such as this: Checking images for similarity with OpenCV . Unfortunately as you can see, my images are not just translated (Rotated/Skewed/Stretched) versions of one another - They actually different images although they are very similar.
You should be able to do it using feature template match function of OpenCV. You can use matchTemplate function to look for the feature and then, minMaxLoc to find its location. Check out the tutorial on OpenCV web site for matchTemplate.
seems you need feature points detections and matching. Check these docs from OpenCV:
http://docs.opencv.org/doc/tutorials/features2d/feature_detection/feature_detection.html
http://docs.opencv.org/doc/tutorials/features2d/feature_flann_matcher/feature_flann_matcher.html
For your particular type of images, you might get good results by using moments/HuMoments for the connected components (which you can find with findContours).
since there is a rotation involved, I dont think template matching would work well. You probably need to use Feature point detection such as SIFT or SURF.
EDIT: This won't work with rotation. Same for matchTemplate. I am yet to try the findContours + moments as in bjoernz answer which sounds promising.
Failed Solution:
I tried using ShapeContextDistanceExtractor(1) available in OpenCV 3.0 along with findContours on your sample images to get good results. The sample images were cropped to same size as original image(128*200). You can could as well use resize in OpenCV.
Code below compares images in images folder with 1.png as the base image.
#include "opencv2/shape.hpp"
#include "opencv2/opencv.hpp"
#include <iostream>
#include <string>
using namespace std;
using namespace cv;
const int MAX_SHAPES = 7;
vector<Point> findContours( const Mat& compareToImg )
{
vector<vector<Point> > contour2D;
findContours(compareToImg, contour2D, RETR_LIST, CHAIN_APPROX_NONE);
//converting 2d vector contours to 1D vector for comparison
vector <Point> contour1D;
for (size_t border=0; border < contour2D.size(); border++) {
for (size_t p=0; p < contour2D[border].size(); p++) {
contour1D.push_back( contour2D[border][p] );
}
}
//limiting contours size to reduce distance comparison time
contour1D.resize( 300 );
return contour1D;
}
int main()
{
string path = "./images/";
cv::Ptr <cv::ShapeContextDistanceExtractor> distanceExtractor = cv::createShapeContextDistanceExtractor();
//base image
Mat baseImage= imread( path + "1.png", IMREAD_GRAYSCALE);
vector<Point> baseImageContours= findContours( baseImage );
for ( int idx = 2; idx <= MAX_SHAPES; ++idx ) {
stringstream imgName;
imgName << path << idx << ".png";
Mat compareToImg=imread( imgName.str(), IMREAD_GRAYSCALE ) ;
vector<Point> contii = findContours( compareToImg );
float distance = distanceExtractor->computeDistance( baseImageContours, contii );
std::cout<<" distance to " << idx << " : " << distance << std::endl;
}
return 0;
}
Result
distance to 2 : 89.7951
distance to 3 : 14.6793
distance to 4 : 6.0063
distance to 5 : 4.79834
distance to 6 : 0.0963184
distance to 7 : 0.00212693
Do three things: 1. Forget about image comparison since you really comparing stroke symbols. 2. Download and play wth a Gesture Search app from google store; 3. Realize that for good performance you cannot recognize your strokes without using timestamp information about stroke drawing. Otherwice we would have a successful handwriting recognition. Then you can research Android stroke reco library to write your code properly.
Related
i'm trying to remove '5 lines' in section, in music papers, my original image is this : http://en.wikipedia.org/wiki/Requiem_(Mozart)#/media/File:K626_Requiem_Mozart.jpg
First, i apply gaussian filter and binarized with threshold (min:100, max 255).
Then applying dft to this image, erase some appropriate lines, and reconstruct image by inverse dft.
i use sample code in opencv documentation, actually i doubt myself that i understand this code. :(
http://docs.opencv.org/doc/tutorials/core/discrete_fourier_transform/discrete_fourier_transform.html
in this sample code, there's 2 Mats. one is 'complexI' for spectrum, another is 'magI' for actual visualized. the result of cv::dft is complexI, magI is normalized complexI. my question is this. how can i add a black line(to cancel in freq domain) and reconstruct?
OpenCV (now) provides a detailed tutorial on how to deal with periodic noise by spectral filtering: https://docs.opencv.org/trunk/d2/d0b/tutorial_periodic_noise_removing_filter.html
It hinges on using cv::dft(), cv::idft(), cv::mulSpectrums(), and cv::magnitude().
The core function (from the tutorial) to perform the filtering goes like so:
void filter2DFreq(const Mat& inputImg, Mat& outputImg, const Mat& H)
{
Mat planes[2] = { Mat_<float>(inputImg.clone()), Mat::zeros(inputImg.size(), CV_32F) };
Mat complexI;
merge(planes, 2, complexI);
// find FT of image
dft(complexI, complexI, DFT_SCALE);
Mat planesH[2] = { Mat_<float>(H.clone()), Mat::zeros(H.size(), CV_32F) };
Mat complexH;
merge(planesH, 2, complexH);
Mat complexIH;
// apply spectral filter
mulSpectrums(complexI, complexH, complexIH, 0);
// reconstruct the filtered image
idft(complexIH, complexIH);
split(complexIH, planes);
outputImg = planes[0];
}
Refer to the tutorial for more information.
Or is it even possible with flann ? Im not the most experienced coder, I also might just be overlooking something really basic (C++,OpenCV 2.4.3.)
The problem :
I have two pointclouds and want to calculate a displacement map. I am trying to use the flann .lib to get the nearest neighbour to a point in the first cloud from the points of the second cloud, and use them and the distance to calculate the displacement vector(s).
What I got so far is this:
int nn = 1;
cv::Mat MyIndex(data1.size(),3,CV_64FC1);
cv::Mat MyQuery(data2.size(),3,CV_64FC1);
cv::Mat indices(data2.size(),1,CV_32SC1);
cv::Mat distances(data2.size(),3,CV_64FC1);
cv::flann::Index_<double> NN_Index(MyIndex, cvflann::KDTreeIndexParams(4));
NN_Index.knnsearch(MyQuery,indices,distances,nn,cvflann::SearchParams(32));
It works as far as I can tell, I got the distances, I got the query points, I got the indices. But how do I get the actual points that got matched to my query points, from the indices ?
I looked through the flann.hpp but couldn't really find any hints. I messed arround a bit with MyIndex, NN_Index and the indices, but didn't get any useful results.
Try
for (int queryIdx = 0; queryIdx < MyQuery.rows; ++queryIdx) {
int dbIdx = indices.at<int>(queryIdx, 0);
std::cout<<"Query Idx:"<<queryIdx<<" matched to "<<"Database Idx:"<<dbIdx<<std::endl;
}
I'm trying to find a measurement for the similarity of 2 faces. I use OpenCV. For that I train Eigenfaces / Fisherfaces with 1000 Photos of 1000 different people (so 1 Photo each person). So I also have 1000 labels in the training set.
Now I can use the predict method to get the most similar face.
I want to input 2 unknown face images to find if they are both similar to the same vector of faces in the training set.
Here is the code of openCV that returns the most similar label (with the lowest distance).
for(size_t sampleIdx = 0; sampleIdx < _projections.size(); sampleIdx++) {
double dist = norm(_projections[sampleIdx], q, NORM_L2);
if((dist < minDist) && (dist < _threshold)) {
minDist = dist;
minClass = _labels.at<int>((int)sampleIdx);
}
Questions:
Can anyone tell me how to rewrite this to output the top 10 faces and not just the top 1 ? I'm thinking about pushing them into a priority queue, but maybe there is something easier?!
In the training: should I put all the faces on the same label or on different labels? So should I have 1 label or 1000 ?
Cheers
Here's what I did. Note I'm really good at perl, really newb at C++ (in fact, this is my first c++ project!) so I output a lot to the command line and parsed it with perl.
I went to facerec.cpp as you did, and I changed the contents of the for loop to this:
for(size_t sampleIdx = 0; sampleIdx < _projections.size(); sampleIdx++) {
double dist = norm(_projections[sampleIdx], q, NORM_L2);
int labelClass = _labels.at<int>((int)sampleIdx);
cout << dist << " " << labelClass << endl;
if((dist < minDist) && (dist < _threshold)) {
minDist = dist;
minClass = _labels.at<int>((int)sampleIdx);
}
}
This now outputs the distance and label of every face. Since all the predict function appears to do is take the picture with the shortest distance (lowest number) and return that as the answer, you can now take the resulting list, sort it, and take the first 10 results. Or you can take the first ten labels or whatever. This just gives you access to all of the data rather than the first X results.
I also added
#include <iostream>
using namespace std;
to the top of the file so I could use cout.
Q1:: Since OpenCV doesn't provide a default function, you have to create your own by creating a vector which has distance and label. You can write your own function as below and store the distance and label in the vector. Here you need to rebuild the opencv.
virtual void predict(InputArray src, int &label, double &confidence, Vector <variable>) const = 0;
I am using an image which holds a marker in a specific area. I tried to do it using Template matching which is the method defined in opencv as cvMatchTemplate.
I am using a web cam to detect them, currently the program is detecting the marker, because I provided the same marker as template.
But I cannot find a way to check whether it is the best match or just slightly matched. Because in cvMatchTemplate it is not only detecting the best match, it also keeps detecting the areas which are slightly matching.
Can any one please tell me a way to do this. Or if there is any other way for my problem, please let me know!
here is the link for my image card
http://imageshack.us/photo/my-images/266/piggycard.jpg/
(I want to detect and check whether its mached)
here is the code
// template_mching_test_2.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include "cv.h"
#include "highgui.h"
int main()
{
IplImage* imgOriginal = cvLoadImage("D:\\4Yr\\Research\\SRS\\Animations\\Piggycard.jpg", 0);
IplImage* imgTemplate = cvLoadImage("D:\\4Yr\\Research\\MakingOf\\Sample Imageas\\PiggyMarkerStart.jpg", 0);
CvCapture *cap = cvCaptureFromCAM(0);
if(!cap)
return -1;
cvNamedWindow("result");
IplImage* imgOriginal;
IplImage* imgOriginal2;
IplImage* imgResult;
while(true)
{
imgOriginal = cvQueryFrame(cap);//cvCreateImage(cvSize(imgOriginal->width-imgTemplate->width+1, imgOriginal->height-imgTemplate->height+1), IPL_DEPTH_32F, 1);
imgOriginal2 = cvCreateImage(cvSize(imgOriginal->width,imgOriginal->height),imgOriginal->depth,1);
imgResult = cvCreateImage(cvSize(imgOriginal->width-imgTemplate->width + 1,imgOriginal->height-imgTemplate->height+1),IPL_DEPTH_32F,1);
cvZero(imgResult);
cvZero(imgOriginal2);
cvCvtColor(imgOriginal,imgOriginal2,CV_BGR2GRAY);
cvMatchTemplate(imgOriginal2, imgTemplate, imgResult,CV_TM_CCORR_NORMED);
double min_val=0, max_val=0;
CvPoint min_loc, max_loc;
cvMinMaxLoc(imgResult, &min_val, &max_val, &min_loc, &max_loc);
cvRectangle(imgOriginal, max_loc, cvPoint(max_loc.x+imgTemplate->width, max_loc.y+imgTemplate->height), cvScalar(0), 1);
printf("%f \n", max_val);
cvShowImage("result", imgOriginal);
cvWaitKey(10);
cvReleaseImage(&imgOriginal2);
cvReleaseImage(&imgResult);
}
cvDestroyAllWindows();
cvReleaseCapture(&cap);
return 0;
}
and as the template I provided the same marker which cropped from the original image. From minMaxLoc i took the max value to check the best match. but it is keep giving me the same values when the image marker in a position, And when the image marker is not in the frame and slightly matching at a place which previous matched with the marker.Does minMaxloc giving us the coordinates(position) of the marker or matching percentage.Or is there any other way for this.
Thank you for your consideration.
There is an OpenCV tutorial on the subject of Template Matching.
Using matchTemplate is a good start, it will provide you with an image containing numbers relating to your matching metric (there is a range of choices for the metric, some of which provide high numbers for better matches, some lower).
To subsequently pick out the best match, you will also need to use the function minMaxLoc which can locate the minimum & maximum values from this matrix.
Currently, I'm working on a project in medical engineering. I have a big image with several sub-images of the cell, so my first task is to divide the image.
I thought about the next thing:
Convert the image into binary
doing a projection of the brightness pixels into the x-axis so I can see where there are gaps between brightnesses values and then divide the image.
The problem comes when I try to reach the second part. My idea is using a vector as the projection and sum all the brightnesses values all along one column, so the position number 0 of the vector is the sum of all the brightnesses values that are in the first column of the image, the same until I reach the last column, so at the end I have the projection.
This is how I have tried:
void calculo(cv::Mat &result,cv::Mat &binary){ //result=the sum,binary the imag.
int i,j;
for (i=0;i<=binary.rows;i++){
for(j=0;j<=binary.cols;j++){
cv::Scalar intensity= binaria.at<uchar>(j,i);
result.at<uchar>(i,i)=result.at<uchar>(i,i)+intensity.val[0];
}
cv::Scalar intensity2= result.at<uchar>(i,i);
cout<< "content" "\n"<< intensity2.val[0] << endl;
}
}
When executing this code, I have a violation error. Another problem is that I cannot create a matrix with one unique row, so...I don't know what could I do.
Any ideas?! Thanks!
At the end, it does not work, I need to sum all the pixels in one COLUMN. I did:
cv::Mat suma(cv::Mat& matrix){
int i;
cv::Mat output(1,matrix.cols,CV_64F);
for (i=0;i<=matrix.cols;i++){
output.at<double>(0,i)=norm(matrix.col(i),1);
}
return output;
}
but It gave me a mistake:
Assertion failed (0 <= colRange.start && colRange.start <= colRange.end && colRange.end <= m.cols) in Mat, file /home/usuario/OpenCV-2.2.0/modules/core/src/matrix.cpp, line 276
I dont know, any idea would be helpful, anyway many thanks mevatron, you really left me in the way.
If you just want the sum of the binary image, you could simply take the L1-norm. Like so:
Mat binaryVectorSum(const Mat& binary)
{
Mat output(1, binary.rows, CV_64F);
for(int i = 0; i < binary.rows; i++)
{
output.at<double>(0, i) = norm(binary.row(i), NORM_L1);
}
return output;
}
I'm at work, so I can't test it out, but that should get you close.
EDIT : Got home. Tested it. It works. :) One caveat...this function works if your binary matrix is truly binary (i.e., 0's and 1's). You may need to scale the norm output with the maximum value if the binary matrix is say 0's and 255's.
EDIT : If you don't have using namespace cv; in your .cpp file, then you'll need to declare the namespace to use NORM_L1 like this cv::NORM_L1.
Have you considered transposing the matrix before you call the function? Like this:
sumCols = binaryVectorSum(binary.t());
vs.
sumRows = binaryVectorSum(binary);
EDIT : A bug with my code :)
I changed:
Mat output(1, binary.cols, CV_64F);
to
Mat output(1, binary.rows, CV_64F);
My test case was a square matrix, so that bug didn't get found...
Hope that is helpful!