OpenCV has a nice in-built ellipse-fitting algorithm called fitEllipse(const Mat& points)
However, it has some major shortcomings, limiting its usefulness. For example, it already requires selected points, so I already have to do a feature extraction myself. HoughCircles detects circles on a given image, pity there is no HoughEllipses.
The other major shortcoming, which stands in the center of my question, is that it does no provide any metric about how accurate the fitting was. It returns an ellipse which best fits the given points, even if the shape does not even remotely look like an ellipse. Is there a way to get the estimated error from the algorithm? I would like to use it as a threshold to filter out shapes which are not even close to be considered ellipses.
I asked this, because maybe there is a simple solution before I try to reinvent the wheel and write my very own fitEllipse function.
If you don't mind getting your hands dirty, you could actually modify the source code for fitEllipse(). The fitEllipse() function uses least-squares to determine the likely ellipses, and the least-squares solution is a tangible distance metric, which is what you want.
If that is something you're willing to do, it would be a very simple code change. Simply add a float whose value is passed back after the function call, where the float stores the current best least-squares value.
fitEllipse gives you the ellipse as a cv::RotatedRect and so you know the angle of rotation of the ellipse, its center and its two axes.
You can compute the sum of the square of the distances between your points and the ellipse, that sum is the metric you are looking for.
The distance between a point and an ellipse is described here http://www.geometrictools.com/Documentation/DistancePointEllipseEllipsoid.pdf and the code is here http://www.geometrictools.com/GTEngine/Include/Mathematics/GteDistPointHyperellipsoid.h
You need to go from OpenCV cv::RotatedRect to Ellipse2 of Geometric Tools Engine and then you can compute the distance.
Why don't you do a findContours() to reduce the memory space required? There's your selected points structure right there. If you want to further simplify you can run a ConvexHull() or ApproxPoly() on that. Fit the ellipse to those points, and then I suppose you can check similarity between the two structures to get some kind of estimate. A difference operator between the two Mats would be a (very) rough estimate?
Depending on the application, you might be able to use CAMShift (or mean shift), which fits an ellipse to a region with similar colors.
Related
I'm looking for an algorithm to find a polygon that can represent a set of points in 2D space. Specifically, if given a set of points like this
It should ideally produce something similar to this:
(The arrows are segments)
Basically, the output would be a set of segments that "best" address the features of the points. The algorithms possibly take some parameters to control the numbers of output segments.
I currently do not have any ideas on what algorithms I'm looking for. Any papers or advice are appreciated.
This is a possible algorithm.
For every point, look at the 2 points closest to it, they become connected.
Then use Douglas Peucker to refine the edges.
Essentially you will create a first polygon containing all the points, and the try to eliminate points whose elimination doesn't change the shape too much.
Is there any opencv function for curve fitting?
I have a set of points (cv::points) and my aim is to fit these points to a closed/open curve.
Right now I am taking a pair of points and drawing lines with them, effectively forming a curve.
It's not quite clear from your question whether you want to smooth the curve by adding more points or to summarise it by using fewer points. If it's the latter, perhaps you should consider cv::approxPolyDP, which is documented here and copied below for reference.
I think you are talking function approximation and interpolation.
As I know, there's not a function directly about curve fitting.
If you just want to get the fitting result, you can use Matlab's curve fitting toolbox, where there is a tool named cftool. cftool is a GUI tool, you can specify the input points and the interpolation method and get the result formula.
I used OpenCV's cv::findHomography API to calculate the homography matrix of two planar images.
The matched key points are extracted by SIFT and matched by BFMatcher. As I know, cv:findHomography use RANSAC iteration to find out the best four corresponding points to get the homography matrix.
So I draw the selected four pairs of points with the calculated contour using homograhy matrix of the edge of the object.
The result are as the links:
https://postimg.cc/image/5igwvfrx9/
As we can see, the selected matched points by RANSAC are correct, but the contour shows that the homography is not accurate.
But these test shows that, both the selected matched points and the homography are correct:
https://postimg.cc/image/dvjnvtm53/
My guess is that if the selected matched points are too close, the small error of the pixel position will lead to the significant error of the homography matrix. If the four points are in the corner of the image, then the shift of the matched points by 4-6 pixels still got good homography matrix.
(According the homogenous coordinate, I think it is reasonable, as the small error in the near plane will be amplified in the far away)
My question is:
1.Is my guess right?
2.Since the four matched points are generated by the RANSAC iteration, the overall error of all the keypoints are minimal. But How to get the stable homography, at least making the contour's mapping is correct? The theory proved that if the four corresponding points in a plane are found, the homography matrix should be calculated, but is there any trick in the engineer work?
I think you're right, and the proximity of the 4 points does not help the accuracy of the result. What you observe is maybe induced by numerical issues: the result may be locally correct for these 4 points but becomes worse when going further.
However, RANSAC will not help you here. The reason is simple: RANSAC is a robust estimation procedure that was designed to find the best point pairs among many correspondences (including some wrong ones). Then, in the inner loop of the RANSAC, a standard homography estimation is performed.
You can see RANSAC as a way to reject wrong point correspondences that would provoke a bad result.
Back to your problem:
What you really need is to have more points. In your examples, you use only 4 point correspondences, which is just enough to estimate an homography.
You will improve your result by providing more matches all over the target image. The problem then becomes over-determined, but a least squares solution can still be found by OpenCV. Furthermore, of there is some error either in the point correspondence process or in some point localization, RANSAC will be able to select the best ones and still give you a reliable result.
If RANSAC results in overfitting on some 4 points (as it seems to be the case in your example), try to relax the constraint by increasing the ransacReprojThreshold parameter.
Alternatively, you can either:
use a different estimator (the robust median CV_LMEDS is a good choice if there are few matching errors)
or use RANSAC in a first step with a large reprojection error (to get a rough estimate) in order to detect the spurious matchings then use LMEDS on the correct ones.
Just to extend #sansuiso's answer, with which I agree:
If you provide around 100 correspondences to RANSAC, probably you are getting more than 4 inliers from cvFindHomography. Check the status output parameter.
To obtain a good homography, you should have many more than 4 correspondences (note that 4 correspondences gives you an homography always), which are well distributed around the image and which are not linear. You can actually use a minimum number of inliers to decide whether the homography obtained is good enough.
Note that RANSAC finds a set of points that are consistent, but the way it has to say that that set is the best one (the reprojection error) is a bit limited. There is a RANSAC-like method, called MSAC, that uses a slightly different error measurement, check it out.
The bad news, in my experience, is that it is little likely to obtain a 100% precision homography most of the times. If you have several similar frames, it is possible that you see that homography changes a little between them.
There are tricks to improve this. For example, after obtaining a homography with RANSAC, you can use it to project your model into the image, and look for new correspondences, so you can find another homography that should be more accurate.
Your target has a lot of symmetric and similar elements. As other people mentioned (and you clarified later) the point spacing and point number can be a problem. Another problem is that SIFT is not designed to deal with significant perspective distortions that are present in your case. Try to track your object through smaller rotations and as was mentioned reproject it using the latest homography to make it look as close as possible to the original. This will also allow you to skip processing heavy SIFT and to use something as lightweight as FAST with cross correlation of image patches for matching.
You also may eventually come to understanding that using points is not enough. You have to use all that you got and this means lines or conics. If a homography transforms a point Pb = H* Pa it is easy to verify that in homogeneous coordinates line Lb = Henv.transposed * La. this directly follows from the equation La’.Pa = 0 = La’ * Hinv * H * Pa = La’ * Hinv * Pb = Lb’.Pb
The possible min. configurations is 1 line and three points or three lines and one point. Two lines and two points doesn’t work. You can use four lines or four points as well. Of course this means that you cannot use the openCV function anymore and has to write your own DLT and then non-linear optimization.
I want to find curvature at depth map
Look at the picture
This is example of curvature
Maybe if i represent image as function and take second derivative from it a can find curvatures. But i couldn't to implement it. (I tryed sobel operator from opencv)
Is there way out?
PS Sorry for my writing mistakes. English in not my native language.
That is not a depth map, it is a point cloud (but I assume it is generated from one single depth map z = f(x,y).
What curvature do you want to estimate? Mean, Gaussian, the whole 2nd fundamental form?
See, e.g. here for definitions. Here's a recent reference on fast estimation methods:
EDIT: I've now found this similar question with a very detailed answer:
proportions of a perspective-deformed rectangle
I'm using OpenCV's findHomography() and warpPerspective() methods to "de skew" a photograph of a sheet of paper. I have this largely working but I'm stuck on a detail.
The part I don't understand how to do is to calculate the optimum set of destination points to input to findHomography(). I know that I want my output to be rectangular, but I dont know the ratio of the width to height of the rectangle. I also want the output rectangle to be sized such that there is minimal scaling of the output image when I apply the transform via warpPerspective(). All I have are the four points that form the quadrilateral I want to transform in the source image. How do I calculate an optimum-sized destination rectangle?
The findHomography() method will need four points (if using Direct Linear Transform). If you want the optimal set you will need the 4-point set which DLT's homography gives the minimum reprojection error. I mean, you need a method that detects inliers/outliers for the particular mathematical model od the DLT.
THis method is RANSAC, and OpenCV has it implemented. You will find examples of findhomography() combined with RANSAC.
I personally find one problem with this and it is the number of iterations of RANSAC in OpenCV, which is too high. If you are looking for optimal speed you will have to dig into the codes.