How can I recognize some basic (usually rotated) shapes:
circle,
"empty" circle,
cross,
empty triangle,
using OpenCV? What's the most straightforward way? It would be nice if the user could "define" his own shapes somehow (load a template image, possibly).
I'd like to map each recognized shape to its type (circle, triangle etc.) and a position of its center. (And its rotation if possible).
For circles HoughCircles could be used (there's no rotation in this case, too). But what about the others? Template matching doesn't support rotation and scaling, right?...
Here's the input:
You are right that regular template matching are not rotation, scale invariant. Take a look at OpenCV's matchShapes. Internally, it uses HuMoments. You will need to use findContours to find each individual object. Now once you have done this, you will probably find matchShapes couldn't distinguish Circle from Ring. A simple way to solve this is to use the hierarchy structure from findContours. If there is a hole (large enough) inside a Circle, that's probably a Ring.
Related
I'd like to register two images with the findTransformECC function offered by OpenCV.
My images have a irregular surrounding border I'd like to mask. I worked with feature based matching functions from the Feature2D-library and findHomography which worked quite well and offered a masking of image parts that should not be taken into account for estimating the transformation parameters.
findTransformECC doesn't offer such a masking, therefore I clipped the images by a centered rectangular shape. The clipped images are aligned very well after transformation. Since I'm using MOTION_EUCLIDEAN - which is just a rotation and translation - I thought I could use exactly the same transformation matrix for aligning the images of original extent - but I was proved wrong. The images aren't correct aligned after transforming them. The orientation of transformed images seem to be OK but images show a wrong translation. My thought was when input-images are clipped with exactly the same centered clipping-area and the rotation is performed around the center the final translation operation should fit as well?
Any suggestions appreciated.
In OpenCV 3.* masking is possible with the findTransformECC function. You can use the argument inputMask in the function.
I want to be able to tell when 2 images collide (not just their frames). But here is the catch: the images are rotating.
So I know how to find whether a pixel in an image is transparent or not but that wont help in this scenario because it will only find the location in the frame relative to a non-rotated image.
Also I have gone as far as trying hit boxes but even those wont work because I can't find a way to detect the collision of UIViews that are contained in different subviews.
Is what I am trying to do even possible?
Thanks in advance
I don't know how you would go about checking for pixel collision on a rotated image. That would be hard. I think you would have to render the rotated image into a context, then fetch pixels from the context to check for transparency. That would be dreadfully slow.
I would suggest a different approach. Come up with a path that maps the bounds of your irregular image. You could then use CGPathContainsPoint to check to see if a set of points is contained in the path (That method takes a transform, which you would use to describe the rotation of your image's path.)
Even then though you're going to have performance problems, since you would have to call that method for a large number of points from the other image to determine if they intersect.
I propose you a simple strategy to solve that, based on looking for rectangles intersections.
The key for that is to create a simplified representation of your images with a set of rectangles laid out properly as bounding boxes of the different part of you image (like you would build your image with legos). For better performance use a small set of rectangles (a few big legos), for better precision use a biggest number of rectangles to precisely follow the image outline.
Your problem becomes equivalent to finding an intersection between rectangles. Or to be more precise to find wether at least one vertex of the rectangles of object A is inside at least one rectangle of object B (CGRectContainsPoint) or if rect intersects (CGRectIntersectsRect).
If you prefer the points lookup, you should define your rectangles by their 4 vertices then it is easy when you rotate your image to apply the same affine transform (use CGPointApplyAffineTransform) to your rectangle vertices to have the coordinates of your points after rotation. But of course you can lookup for frame intersections and represent you rectangle using the standard CGRect structure.
You could also use a CGPath (as explained in another answer below) instead of a set of rectangles and look for any vertex inside other path using CGPathContainsPoint. That would give the same result actually but probably the rectangles approach is faster in many cases.
The only trick is to take one of the objects as a reference axis. Imagine you are on object A and you only see B moving around you. Then if you have to rotate A you need to make an axis transform to always have B transform relatively to A and not to the screen or any other reference. If your transforms are only rotation around the object centre then rotating A by n radians is equivalent to rotating B by -n radians.
Then just loop through your vertices defining object A and find if one is inside a rectangle of object A.
You will probably need to investigate a bit to achieve that but at least you have some clues on how to solve that.
I am currently looking for a way to fit a simple shape (e.g. a T or an L shape) to a 2D point cloud. What I need as a result is the position and orientation of the shape.
I have been looking at a couple of approaches but most seem very complicated and involve building and learning a sample database first. As I am dealing with very simple shapes I was hoping that there might be a simpler approach.
By saying you don't want to do any training I am guessing that you mean you don't want to do any feature matching; feature matching is used to make good guesses about the pose (location and orientation) of the object in the image, and would be applicable along with RANSAC to your problem for guessing and verifying good hypotheses about object pose.
The simplest approach is template matching, but this may be too computationally complex (it depends on your use case). In template matching you simply loop over the possible locations of the object and its possible orientations and possible scales and check how well the template (a cloud that looks like an L or a T at that location and orientation and scale) matches (or you sample possible locations orientations and scales randomly). The checking of the template could be made fairly fast if your points are organised (or you organise them by e.g. converting them into pixels).
If this is too slow there are many methods for making template matching faster and I would recommend to you the Generalised Hough Transform.
Here, before starting the search for templates you loop over the boundary of the shape you are looking for (T or L) and for each point on its boundary you look at the gradient direction and then the angle at that point between the gradient direction and the origin of the object template, and the distance to the origin. You add that to a table (Let us call it Table A) for each boundary point and you end up with a table that maps from gradient direction to the set of possible locations of the origin of the object. Now you set up a 2D voting space, which is really just a 2D array (let us call it Table B) where each pixel contains a number representing the number of votes for the object in that location. Then for each point in the target image (point cloud) you check the gradient and find the set of possible object locations as found in Table A corresponding to that gradient, and then add one vote for all the corresponding object locations in Table B (the Hough space).
This is a very terse explanation but knowing to look for Template Matching and Generalised Hough transform you will be able to find better explanations on the web. E.g. Look at the Wikipedia pages for Template Matching and Hough Transform.
You may need to :
1- extract some features from the image inside which you are looking for the object.
2- extract another set of features in the image of the object
3- match the features (it is possible using methods like SIFT)
4- when you find a match apply RANSAC algorithm. it provides you with transformation matrix (including translation, rotation information).
for using SIFT start from here. it is actually one of the best source-codes written for SIFT. It includes RANSAC algorithm and you do not need to implement it by yourself.
you can read about RANSAC here.
Two common ways for detecting the shapes (L, T, ...) in your 2D pointcloud data would be using OpenCV or Point Cloud Library. I'll explain steps you may take for detecting those shapes in OpenCV. In order to do that, you can use the following 3 methods and the selection of the right method depends on the shape (Size, Area of the shape, ...):
Hough Line Transformation
Template Matching
Finding Contours
The first step would be converting your point to a grayscale Mat object, by doing that you basically make an image of your 2D pointcloud data and so you can use other OpenCV functions. Then you may smooth the image in order to reduce the noises and the result would be somehow a blurry image which contains real edges, if your application does not need real-time processing, you can use bilateralFilter. You can find more information about smoothing here.
The next step would be choosing the method. If the shape is just some sort of orthogonal lines (such as L or T) you can use Hough Line Transformation in order to detect the lines and after detection, you can loop over the lines and calculate the dot product of the lines (since they are orthogonal the result should be 0). You can find more information about Hough Line Transformation here.
Another way would be detecting your shape using Template Matching. Basically, you should make a template of your shape (L or T) and use it in matchTemplate function. You should consider that the size of the template you want to use should be in the order of your image, otherwise you may resize your image. More information about the algorithm can be found here.
If the shapes include areas you can find contours of the shape using findContours, it will give you the number of polygons which are around your shape you want to detect. For instance, if your shape is L, it would have polygon which has roughly 6 lines. Also, you can use some other filters along with findContours such as calculating the area of the shape.
I am try to draw shapes in the window in real time. The shapes are like tangle ,rectangle ,circle , half or circle and "Z" in the screen using yellow color. The size and the shape may not be same to the original image. But Program know all the original shapes. Because they are predefined. I want to know how i can identify the correct shape. as an example,
is there possible way to do this? can I use template matching for this? Please help me with this..
You can use different methods to detect each shape. Check these:
Ellipse detection with OpenCV
Square detection doesn't find squares
I want to identify an object and draw a shape around it ...
I used previously the color identification but wasn't a good option since color change dramatically from place to place .. so I though why not identifying objects by features such as edges .. and I did that using this function in openCV
cvgoodfeaturesTotrack
it returns the (x,y)-coordinates of the points .. now I want to connect those points.. well not all of them but the one who are close to each other to draw a shape around the different objects. Any ideas ?
I don't think there is a free lunch in this case. You are trying to reconstruct a polygon if you only know the corner points of the polygon. There is no unique solution to this problem: you can draw all sorts of polygons through the corners. If you are certain the shape you are after is convex, then you can construct the convex span of the corner points, but the result will be horrible if you include any corners that were not part of the original object.
It seems to me that detecting corners is not the way to segment an object that is more or less delimited by lines. You probably want to try an edge detector instead, or a proper segmentation technique such as watershed.