functions in opencv programming - opencv

I am a newbie to opencv and am trying to implement shape context descriptor outlined in the slide http://www.cs.utexas.edu/~grauman/courses/spring2008/slides/ShapeContexts425.pdf
I found the edge points on the shape using canny edge detector for the first part of step 1. Then I need to calculate the Euclidean distance on each edge point to the other ones. Rather than using for-loops to find the distance between each and every point, is there any opencv function that can do this step more efficiently?

Finding all pairwise distances between the set of points is not a standard operation and I don't think you will find something similar in OpenCV. And it is very easy to compute by hand. Given two points a and b, you can calculate distance between them as cv::norm(a - b), as described here.
You might want to utilize the matchShapes function. However, it operates with image moments, not with shape descriptor you mentioned.

Related

OpenCV Edge Match

I am pretty new to opencv and I have read some tutorials and projects source code. It seems that people tend to do the object detection base on color.
I wonder if there is a way to do the object detection base on the curve of the edge.
For example, I have a known object whose edge is like A, and I have known that the edge of this object in different images is pretty similar but not the same, like B. Now consider B is in the Pic C, and my target is to find out the contour or edge of B in the C based on the searching for the similar shape of A in the C.
So is there any method of contour detection or edge detection based on the searching for the similar shape of an known shape.
I think the default method in OpenCV is matchShapes; but I have used a generic method that works rather well.
For each contour, take the centroid.
Following the contour from the point highest above the centroid, take the distance to the centroid.
Move a step along the contour (for example 1/100 of the length of the contour). The the distance again.
After all the steps, you have a arrays of distances.
Compare arrays, for example using Cosine similarity
You can make this method scale and rotation invariant.
If you want to have scale invariance, normalize the array before comparison.
If you want to have rotation invariance, you can shift the array and find the best match over the shifst

curve fitting in OpenCV

Is there any opencv function for curve fitting?
I have a set of points (cv::points) and my aim is to fit these points to a closed/open curve.
Right now I am taking a pair of points and drawing lines with them, effectively forming a curve.
It's not quite clear from your question whether you want to smooth the curve by adding more points or to summarise it by using fewer points. If it's the latter, perhaps you should consider cv::approxPolyDP, which is documented here and copied below for reference.
I think you are talking function approximation and interpolation.
As I know, there's not a function directly about curve fitting.
If you just want to get the fitting result, you can use Matlab's curve fitting toolbox, where there is a tool named cftool. cftool is a GUI tool, you can specify the input points and the interpolation method and get the result formula.

Metric for ellipse fitting in OpenCV

OpenCV has a nice in-built ellipse-fitting algorithm called fitEllipse(const Mat& points)
However, it has some major shortcomings, limiting its usefulness. For example, it already requires selected points, so I already have to do a feature extraction myself. HoughCircles detects circles on a given image, pity there is no HoughEllipses.
The other major shortcoming, which stands in the center of my question, is that it does no provide any metric about how accurate the fitting was. It returns an ellipse which best fits the given points, even if the shape does not even remotely look like an ellipse. Is there a way to get the estimated error from the algorithm? I would like to use it as a threshold to filter out shapes which are not even close to be considered ellipses.
I asked this, because maybe there is a simple solution before I try to reinvent the wheel and write my very own fitEllipse function.
If you don't mind getting your hands dirty, you could actually modify the source code for fitEllipse(). The fitEllipse() function uses least-squares to determine the likely ellipses, and the least-squares solution is a tangible distance metric, which is what you want.
If that is something you're willing to do, it would be a very simple code change. Simply add a float whose value is passed back after the function call, where the float stores the current best least-squares value.
fitEllipse gives you the ellipse as a cv::RotatedRect and so you know the angle of rotation of the ellipse, its center and its two axes.
You can compute the sum of the square of the distances between your points and the ellipse, that sum is the metric you are looking for.
The distance between a point and an ellipse is described here http://www.geometrictools.com/Documentation/DistancePointEllipseEllipsoid.pdf and the code is here http://www.geometrictools.com/GTEngine/Include/Mathematics/GteDistPointHyperellipsoid.h
You need to go from OpenCV cv::RotatedRect to Ellipse2 of Geometric Tools Engine and then you can compute the distance.
Why don't you do a findContours() to reduce the memory space required? There's your selected points structure right there. If you want to further simplify you can run a ConvexHull() or ApproxPoly() on that. Fit the ellipse to those points, and then I suppose you can check similarity between the two structures to get some kind of estimate. A difference operator between the two Mats would be a (very) rough estimate?
Depending on the application, you might be able to use CAMShift (or mean shift), which fits an ellipse to a region with similar colors.

Finding simple shapes in 2D point clouds

I am currently looking for a way to fit a simple shape (e.g. a T or an L shape) to a 2D point cloud. What I need as a result is the position and orientation of the shape.
I have been looking at a couple of approaches but most seem very complicated and involve building and learning a sample database first. As I am dealing with very simple shapes I was hoping that there might be a simpler approach.
By saying you don't want to do any training I am guessing that you mean you don't want to do any feature matching; feature matching is used to make good guesses about the pose (location and orientation) of the object in the image, and would be applicable along with RANSAC to your problem for guessing and verifying good hypotheses about object pose.
The simplest approach is template matching, but this may be too computationally complex (it depends on your use case). In template matching you simply loop over the possible locations of the object and its possible orientations and possible scales and check how well the template (a cloud that looks like an L or a T at that location and orientation and scale) matches (or you sample possible locations orientations and scales randomly). The checking of the template could be made fairly fast if your points are organised (or you organise them by e.g. converting them into pixels).
If this is too slow there are many methods for making template matching faster and I would recommend to you the Generalised Hough Transform.
Here, before starting the search for templates you loop over the boundary of the shape you are looking for (T or L) and for each point on its boundary you look at the gradient direction and then the angle at that point between the gradient direction and the origin of the object template, and the distance to the origin. You add that to a table (Let us call it Table A) for each boundary point and you end up with a table that maps from gradient direction to the set of possible locations of the origin of the object. Now you set up a 2D voting space, which is really just a 2D array (let us call it Table B) where each pixel contains a number representing the number of votes for the object in that location. Then for each point in the target image (point cloud) you check the gradient and find the set of possible object locations as found in Table A corresponding to that gradient, and then add one vote for all the corresponding object locations in Table B (the Hough space).
This is a very terse explanation but knowing to look for Template Matching and Generalised Hough transform you will be able to find better explanations on the web. E.g. Look at the Wikipedia pages for Template Matching and Hough Transform.
You may need to :
1- extract some features from the image inside which you are looking for the object.
2- extract another set of features in the image of the object
3- match the features (it is possible using methods like SIFT)
4- when you find a match apply RANSAC algorithm. it provides you with transformation matrix (including translation, rotation information).
for using SIFT start from here. it is actually one of the best source-codes written for SIFT. It includes RANSAC algorithm and you do not need to implement it by yourself.
you can read about RANSAC here.
Two common ways for detecting the shapes (L, T, ...) in your 2D pointcloud data would be using OpenCV or Point Cloud Library. I'll explain steps you may take for detecting those shapes in OpenCV. In order to do that, you can use the following 3 methods and the selection of the right method depends on the shape (Size, Area of the shape, ...):
Hough Line Transformation
Template Matching
Finding Contours
The first step would be converting your point to a grayscale Mat object, by doing that you basically make an image of your 2D pointcloud data and so you can use other OpenCV functions. Then you may smooth the image in order to reduce the noises and the result would be somehow a blurry image which contains real edges, if your application does not need real-time processing, you can use bilateralFilter. You can find more information about smoothing here.
The next step would be choosing the method. If the shape is just some sort of orthogonal lines (such as L or T) you can use Hough Line Transformation in order to detect the lines and after detection, you can loop over the lines and calculate the dot product of the lines (since they are orthogonal the result should be 0). You can find more information about Hough Line Transformation here.
Another way would be detecting your shape using Template Matching. Basically, you should make a template of your shape (L or T) and use it in matchTemplate function. You should consider that the size of the template you want to use should be in the order of your image, otherwise you may resize your image. More information about the algorithm can be found here.
If the shapes include areas you can find contours of the shape using findContours, it will give you the number of polygons which are around your shape you want to detect. For instance, if your shape is L, it would have polygon which has roughly 6 lines. Also, you can use some other filters along with findContours such as calculating the area of the shape.

calculating the destination points for OpenCV's findHomography

EDIT: I've now found this similar question with a very detailed answer:
proportions of a perspective-deformed rectangle
I'm using OpenCV's findHomography() and warpPerspective() methods to "de skew" a photograph of a sheet of paper. I have this largely working but I'm stuck on a detail.
The part I don't understand how to do is to calculate the optimum set of destination points to input to findHomography(). I know that I want my output to be rectangular, but I dont know the ratio of the width to height of the rectangle. I also want the output rectangle to be sized such that there is minimal scaling of the output image when I apply the transform via warpPerspective(). All I have are the four points that form the quadrilateral I want to transform in the source image. How do I calculate an optimum-sized destination rectangle?
The findHomography() method will need four points (if using Direct Linear Transform). If you want the optimal set you will need the 4-point set which DLT's homography gives the minimum reprojection error. I mean, you need a method that detects inliers/outliers for the particular mathematical model od the DLT.
THis method is RANSAC, and OpenCV has it implemented. You will find examples of findhomography() combined with RANSAC.
I personally find one problem with this and it is the number of iterations of RANSAC in OpenCV, which is too high. If you are looking for optimal speed you will have to dig into the codes.

Resources