i have a code like this:
Mat img = Highgui.imread(inFile);
Mat templ = Highgui.imread(templateFile);
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
Imgproc.matchTemplate(img, templ, result, Imgproc.TM_CCOEFF);
/////Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
for (int i = 0; i < result_rows; i++)
for (int j = 0; j < result_cols; j++)
if(result.get(i, j)[0]>?)
//match!
I need to parse the input image to find multiple occurrencies of the template image. I want to have a result like this:
result[0][0]= 15%
result[0][1]= 17%
result[x][y]= 47%
If i use TM_COEFF all results are [-xxxxxxxx.xxx,+xxxxxxxx.xxx]
If i use TM_SQDIFF all results are xxxxxxxx.xxx
If i use TM_CCORR all results are xxxxxxxx.xxx
How can i detect a match or a mismatch? What is the right condition into the if?
If i normalized the matrix the application set a value to 1 and i can't detect if the template isn't stored into the image (all mismatch).
Thanks in advance
You can append "_NORMED" to the method names (For instance: CV_TM_COEFF_NORMED in C++; could be slightly different in Java) to get a sensible value for your purpose.
By 'sensible', I mean that you will get values in the range of 0 to 1 which can be multiplied by 100 for your purpose.
Note: For CV_TM_SQDIFF_NORMED, it will be in the range -1 to 0, and you will have to subtract the value from 1 in order to make sense of it, because the lowest value if used in this method.
Tip: you can use the java equivalent of minMaxLoc() in order to get the minimum and maximum values. It's very useful when used in conjunction with matchtemplate.
I believe 'minMaxLoc' that is located inside the class Core.
Here's a C++ implementation:
matchTemplate( input_mat, template_mat, result_mat, method_NORMED );
double minVal, maxVal;
double percentage;
Point minLoc; Point maxLoc;
minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
if( method_NORMED == CV_TM_SQDIFF_NORMED )
{
percentage=1-minVal;
}
else
{
percentage=maxVal;
}
Useful C++ docs:
Match template description along with available methods: http://docs.opencv.org/modules/imgproc/doc/object_detection.html
MinMaxLoc documentation:
http://docs.opencv.org/modules/core/doc/operations_on_arrays.html?highlight=minmaxloc#minmaxloc
Another approach will be background differencing. You can observe the distortion.
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.highgui.Highgui;
import org.opencv.imgproc.Imgproc;
public class BackgroundDifference {
public static void main(String[] arg){
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
Mat model = Highgui.imread("e:\\answers\\template.jpg",Highgui.CV_LOAD_IMAGE_GRAYSCALE);
Mat scene = Highgui.imread("e:\\answers\\front7.jpg",Highgui.CV_LOAD_IMAGE_GRAYSCALE);
Mat diff = new Mat();
Core.absdiff(model,scene,diff);
Imgproc.threshold(diff,diff,15,255,Imgproc.THRESH_BINARY);
int distortion = Core.countNonZero(diff);
System.out.println("distortion:"+distortion);
Highgui.imwrite("e:\\answers\\diff.jpg",diff);
}
}
Related
I´m trying to transform a point from one map to another. I´ve tried to use some OpenCV sample code for getAffineTransform(), getPerspectiveTransform(), warpAffine() and findHomography(), but there´re always some kind of gaps in my transformation mesh. The feature points are usually detected on very different positions, so I need a good interpolation method, I think.
About the maps:
Both maps are images which are containing human body parts and human skin. I´m using the OpenCV feature detection/matching algorithmns to get a couple of equal points in both maps. The tricky thing is they´re containing arms and feets, too. Feature points on arms/feets can have much bigger offsets than the points on the torso.
The goal:
I want to transform any point on map A as good as possible to the equivalent position on map B.
My current approach is to find the three most clostest points to my original point on map A and construct a triangle. Afterwards I transform this triangle to the same three feature points on map B. That´s working nice if I have a lot of close feature point surrounding my original point. But on larger areas without feature points I got some problems with the interpolation.
Is this a good way to do so? Or is there a much better solution?
My favorite one would be the contruction of a complete transformation map for both images, but I´m not sure how to do this. Is it possible at all?
Thanks a lot for any advice!
Simple sketch of the transformation (I´m trying to find the points X1 to X3 from the left image in the right image):
Sketch of a sample transformation
Sample for homography (OpenCVSharp):
Mat imgA = new Mat(#"d:\Mesh\Left2.jpg", ImreadModes.Color);
Mat imgB = new Mat(#"d:\Mesh\Right2.jpg", ImreadModes.Color);
Cv2.Resize(imgA, imgA, new Size(512, 341));
Cv2.Resize(imgB, imgB, new Size(512, 341));
SURF detector = SURF.Create(500.0);
KeyPoint[] keypointsA = detector.Detect(imgA);
KeyPoint[] keypointsB = detector.Detect(imgB);
SIFT extractor = SIFT.Create();
Mat descriptorsA = new Mat();
Mat descriptorsB = new Mat();
extractor.Compute(imgA, ref keypointsA, descriptorsA);
extractor.Compute(imgB, ref keypointsB, descriptorsB);
BFMatcher matcher = new BFMatcher(NormTypes.L2, true);
DMatch[] matches = matcher.Match(descriptorsA, descriptorsB);
double minDistance = 10000.0;
double maxDistance = 0.0;
for (int i = 0; i < matches.Length; ++i)
{
double distance = matches[i].Distance;
if (distance < minDistance)
{
minDistance = distance;
}
if (distance > maxDistance)
{
maxDistance = distance;
}
}
List<DMatch> goodMatches = new List<DMatch>();
for (int i = 0; i < matches.Length; ++i)
{
if (matches[i].Distance <= 3.0 * minDistance &&
Math.Abs(keypointsA[matches[i].QueryIdx].Pt.Y - keypointsB[matches[i].TrainIdx].Pt.Y) < 30)
{
goodMatches.Add(matches[i]);
}
}
Mat output = new Mat();
Cv2.DrawMatches(imgA, keypointsA, imgB, keypointsB, goodMatches.ToArray(), output);
List<Point2f> goodA = new List<Point2f>();
List<Point2f> goodB = new List<Point2f>();
for (int i = 0; i < goodMatches.Count; i++)
{
goodA.Add(keypointsA[goodMatches[i].QueryIdx].Pt);
goodB.Add(keypointsB[goodMatches[i].TrainIdx].Pt);
}
InputArray goodInputA = InputArray.Create<Point2f>(goodA);
InputArray goodInputB = InputArray.Create<Point2f>(goodB);
Mat h = Cv2.FindHomography(goodInputA, goodInputB);
Point2f centerA = new Point2f(imgA.Cols / 2.0f, imgA.Rows / 2.0f);
output.DrawMarker((int)centerA.X, (int)centerA.Y, Scalar.Red, MarkerStyle.Cross, 50, LineTypes.Link8, 5);
Point2f[] transformedPoints = Cv2.PerspectiveTransform(new Point2f[] { centerA }, h);
output.DrawMarker((int)transformedPoints[0].X + imgA.Cols, (int)transformedPoints[0].Y, Scalar.Red, MarkerStyle.Cross, 50, LineTypes.Link8, 5);
Code snippet for perspective transform (different approach, OpenCVSharp):
pointsA[0] = new Point(trisA[i].Item0, trisA[i].Item1);
pointsA[1] = new Point(trisA[i].Item2, trisA[i].Item3);
pointsA[2] = new Point(trisA[i].Item4, trisA[i].Item5);
pointsB[0] = new Point(trisB[i].Item0, trisB[i].Item1);
pointsB[1] = new Point(trisB[i].Item2, trisB[i].Item3);
pointsB[2] = new Point(trisB[i].Item4, trisB[i].Item5);
Mat transformation = Cv2.GetAffineTransform(pointsA, pointsB);
InputArray inputSource = InputArray.Create<Point2f>(new Point2f[] { new Point2f(10f, 50f) });
Mat outputMat = new Mat();
Cv2.PerspectiveTransform(inputSource, outputMat, transformation);
Mat.Indexer<Point2f> indexer = outputMat.GetGenericIndexer<Point2f>();
var target = indexer[0, 0];
I am new to OpenCV and I want to select a particular region in the video/image for detection. In my case I want to detect cars that are only in the road not in the parking lot.
Well, selecting cars requires use of training data. But to select an ROI (region of interest) is fairly simple:
Consider img = cv2.imread(image)
In that case, somewhere in your code, you can specify a region this way:
sub_image = img[y:y+h, x:x+w]
That will get the ROI once you specify the values, of course, not using 'x' or 'y', where h is the height and w is the width. Remember that images are just 2D matrices.
Use CascadeClassifier() to select the car(s) from the image(s). Documentation is found here. OpenCV comes packed with training data you can use to make classifications in the form of XML files.
If you want to manually select a region of interest (ROI) to do some processing on it, then you may trying using mouse click event to select start and stop points of your ROI.
Once you have start and stop point you can use it to retrieve image from selected region.
The can be done on image or capture video frame.
bool roi_captured = false;
Point pt1, pt2;
Mat cap_img;
//Callback for mousclick event, the x-y coordinate of mouse button-up and button-down
//are stored in two points pt1, pt2.
void mouse_click(int event, int x, int y, int flags, void *param)
{
switch(event)
{
case CV_EVENT_LBUTTONDOWN:
{
std::cout<<"Mouse Pressed"<<std::endl;
if(!roi_capture)
{
pt1.x = x;
pt1.y = y;
}
else
{
std::cout<<"ROI Already Acquired"<<std::endl;
}
break;
}
case CV_EVENT_LBUTTONUP:
{
if(!got_roi)
{
Mat cl;
std::cout<<"Mouse LBUTTON Released"<<std::endl;
pt2.x = x;
pt2.y = y;
cl = cap_img.clone();
Mat roi(cl, Rect(pt1, pt2));
Mat prev_imgT = roi.clone();
std::cout<<"PT1"<<pt1.x<<", "<<pt1.y<<std::endl;
std::cout<<"PT2"<<pt2.x<<","<<pt2.y<<std::endl;
imshow("Clone",cl);
got_roi = true;
}
else
{
std::cout<<"ROI Already Acquired"<<std::endl;
}
break;
}
}
}
//In main open video and wait for roi event to complete by the use.
// You capture roi in pt1 and pt2 you can use the same coordinates for processing // //subsequent frame
int main(int argc, char *argv[])
{
int frame_num = 0;
int non_decode_frame =0;
int count = 1, idx =0;
int frame_pos =0;
std::cout<<"Video File "<<argv[1]<<std::endl;
cv::VideoCapture input_video(argv[1]);
namedWindow("My_Win",1);
cvSetMouseCallback("My_Win", mouse_click, 0);
sleep(1);
while(input_video.grab())
{
cap_img.release();
if(input_video.retrieve(cap_img))
{
imshow("My_Win", cap_img);
if(!got_roi)
{
//Wait here till user select the desire ROI
waitKey(0);
}
else
{
std::cout<<"Got ROI disp prev and curr image"<<std::endl;
std::cout<<"PT1"<<pt1.x<<" "<<pt1.y<<std::endl;
std::cout<<"PT2"<<pt2.x<<" "<<pt2.y<<std::endl;
Mat curr_img_t1;
Mat roi2(cap_img,Rect(pt1, pt2));
Mat curr_imgT = roi2.clone();
cvtColor(curr_imgT, curr_img_t1, CV_RGB2GRAY);
imshow("curr_img", curr_img);
// Do remaining processing here on capture roi for every frame
waitKey(1);
}
}
}
}
You didn't tag in what programming language you are writing with. Anyway, I answer you in python. (You can easily convert it to C++ if you want)
def mouse_drawing(event, x, y, flags, params):
if event == cv2.EVENT_LBUTTONDOWN:
car = img[y: y + carheight, x: x + carwidth]
cv2.imwrite("car", car)
cv2.namedWindow("my_img")
cv2.setMouseCallback("my_img", mouse_drawing)
while True:
cv2.imshow("my_img", img)
key = cv2.waitKey(1)
if key == 27:
break
As in other answers was told, if you want to find cars automatically, that would be another problem and has to do with training data and other things.
I am in need of using the standard Hough Transformation (instead of the using the HoughLinesBinary method which implements Probabilistic Hough Transform) and have attempted doing so by creating a custom version of the HoughLinesBinary method:
using (MemStorage stor = new MemStorage())
{
IntPtr lines = CvInvoke.cvHoughLines2(canny.Ptr, stor.Ptr, Emgu.CV.CvEnum.HOUGH_TYPE.CV_HOUGH_STANDARD, rhoResolution, (thetaResolution*Math.PI)/180, threshold, 0, 0);
Seq<MCvMat> segments = new Seq<MCvMat>(lines, stor);
List<MCvMat> lineslist = segments.ToList();
foreach(MCvMat line in lineslist)
{
//Process lines: (rho, theta)
}
}
My problem is that I am unsure of what type is the sequence returned. I believe it should be MCvMat, due to reading the documentation that CvMat* is used in OpenCV, which also states that for STANDARD "the matrix must be (the created sequence will be) of CV_32FC2 type"
I am unclear as to what I would need to do to return and process that correct output data from the STANDARD hough lines (i.e. the 2x1 vector for each line giving the rho and theta information).
Any help would be greatly appreciated. Thank you
-Sal
I had the same problem myself a couple of days ago. This is how I solved it using marshalling. Please let me know if you find a simpler solution.
using (MemStorage stor = new MemStorage())
{
IntPtr lines = CvInvoke.cvHoughLines2(canny.Ptr, stor.Ptr, Emgu.CV.CvEnum.HOUGH_TYPE.CV_HOUGH_STANDARD, rhoResolution, (thetaResolution*Math.PI)/180, threshold, 0, 0);
int maxLines = 100;
for(int i = 0; i < maxLines; i++)
{
IntPtr line = CvInvoke.cvGetSeqElem(lines, i);
if (line == IntPtr.Zero)
{
// No more lines
break;
}
PolarCoordinates coords = (PolarCoordinates)System.Runtime.InteropServices.Marshal.PtrToStructure(line, typeof(PolarCoordinates));
// Do something with your Hough lines
}
}
with a struct defined as follows:
public struct PolarCoordinates
{
public float Rho;
public float Theta;
}
I am passing a Mat to another function and changing it inside the called function. I had expected that being a more complex type it was automatically passed by reference so that the matrix would have changed in the calling function, but it doesn't. Could someone point me at the explanation of how to correctly return a changed Mat from a function?
Here's the code snippet:
void callingFunction(Mat img)
{
Mat tst(100,500,CV_8UC3, Scalar(0,255,0));
saveImg(tst, "Original image", true);
testImg(tst);
saveImg(tst, "Want it to be same as inside testImg but is same as Original", true);
}
void testImg(Mat img)
{
int rs = 50; // rows
int cs = 100; // columns
img = Mat(rs, cs, CV_8UC3, Scalar(255,0,0));
Mat roi(img, Rect(0, 0, cs, rs/2));
roi = Scalar(0,0,255); // change a subsection to a different color
saveImg(img, "inside testImg", true);
}
Thanks!
You have to define Mat as parameter-reference (&). Here's edited code:
void callingFunction(Mat& img)
{
Mat tst(100,500,CV_8UC3, Scalar(0,255,0));
saveImg(tst, "Original image", true);
testImg(tst);
saveImg(tst, "Want it to be same as inside testImg but is same as Original", true);
}
void testImg(Mat& img)
{
int rs = 50; // rows
int cs = 100; // columns
img = Mat(rs, cs, CV_8UC3, Scalar(255,0,0));
Mat roi(img, Rect(0, 0, cs, rs/2));
roi = Scalar(0,0,255); // change a subsection to a different color
saveImg(img, "inside testImg", true);
}
I wondered about the same question myself, so I would like to further clarify the answer given by #ArtemStorozhuk (which is correct).
The OpenCV documentation is misleading here, because it appears you're passing the matrix by value, but in fact the constructor of cv::OutputArray is defined as follows:
_OutputArray::_OutputArray(Mat& m)
so it gets the matrix by reference!
Since operations like cv::Mat::create create a new matrix, the operation releases the reference and set the couter to 1. Thus, in order to keep the result in the calling function, you have to pass the matrix by reference.
If its true that you have to explicitly pass by reference, then how do all the OpenCV functions work? None of them pass values by reference, yet they somehow seem to write to the passed in Mat just fine. For example, here is the declaration for the Sobel function in imgproc.hpp:
//! applies generalized Sobel operator to the image
CV_EXPORTS_W void Sobel( InputArray src, OutputArray dst, int ddepth,
int dx, int dy, int ksize=3,
double scale=1, double delta=0,
int borderType=BORDER_DEFAULT );
as you can see, it passes in src and dst without a &. And yet I know that after I call the Sobel with an empty dst, it will end up filled. No '&' involved.
After spending a couple of days trying figure out why opencv DFT would give 100% similar results for all three channels I ended up finding out that there might be a bug in the split() function that OpenCV provides for splitting a input image to 3 single channel images.
std::vector<cv::Mat> rgbChannels(3,cv::Mat(inputImage.size(),CV_64FC1));
cv::split(inputImage, rgbChannels);
After saving the image values onto disk and using a file differencing tool, I found out that all values in the split channels were identical.
Have I done something wrong?
My work around was the following function. But that also gave me the exact identical values, giving me a hint that somehow vectors were not being handled correctly by OpenCV.
SplitImage(cv::Mat inputImage)
{
//copy original in BGR order
std::vector<cv::Mat> splittedImage(3,cv::Mat(inputImage.size(),CV_64FC1));
cv::Mat tempImage(inputImage.size(),CV_64FC1);
for (int row = 0; row < inputImage.size().height; row++)
{
for (int col = 0; col < inputImage.size().width; col++)
{
splittedImage[0].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[0];
splittedImage[1].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[1];
splittedImage[2].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[2];
}
}
return splittedImage;
}
And finally wrote the following to solve the problem
SplitImage(cv::Mat inputImage)
{
//copy original in BGR order
std::vector<cv::Mat> splittedImage(3,cv::Mat(inputImage.size(),CV_64FC1));
std::vector<cv::Mat>::iterator it;
it = splittedImage.begin();
for(int channelNo = 0; channelNo < inputImage.channels(); channelNo++)
{
cv::Mat tempImage(inputImage.size(),CV_64FC1);
for (int row = 0; row < inputImage.size().height; row++)
{
for (int col = 0; col < inputImage.size().width; col++)
{
tempImage.at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[channelNo];
}
}
it = splittedImage.insert ( it , tempImage );
it++;
}
return splittedImage;
}
Has anyone had a problem with split() function or have I done something wrong?
It is not a bug in OpenCV but there is a problem with your code.
The following line does not create a vector of 3 different Mats:
std::vector<cv::Mat> rgbChannels(3,cv::Mat(inputImage.size(),CV_64FC1));
Instead, this line produces a vector of 3 Mat headers sharing the same memory. It works this way because Mat copy constructor does not make a deep copy - it just increments an internal reference counter.
Just change your code to the following to solve your problem:
std::vector<cv::Mat> rgbChannels(3);
cv::split(inputImage, rgbChannels);