I want algorithm for recognizing multiple no of shapes(Specially rectangle and squares) in a picture.Preferably I am using C# so, I am looking forward for solutions in C#.
check aforgenet....
http://www.aforgenet.com/forum/
If you are looking for a library that does a lot of image processing for you there is always OpenCV. I think it is is c++ though.
You can use the Circularity algorithm as a first approach, which is very easy to compute:
C = p2/a where p is the perimeter (border area) and a is shape area.
To know how to read/write pixels quickly, take a look here
Alternatively look for shape signature algorithm available at Rafael Gonzales book. In this algorithm you compute the center of the object using central momentum, the you compute the distance between the center and each border pixel. You'll end up with a 1D signal where peaks represent bigger distance from the center. In a square, you have 4 symmetric peaks while in a rectangle 2 big peaks and 2 smaller ones.
Related
I have an array of data from a grayscale image that I have segmented sets of contiguous points of a certain intensity value from.
Currently I am doing a naive bounding box routine where I find the minimum and maximum (x,y) [row, col] points. This obviously does not provide the smallest possible box that contains the set of points which is demonstrable by simply rotating a rectangle so the longest axis is no longer aligned with a principal axis.
What I wish to do is find the minimum sized oriented bounding box. This seems to be possible using an algorithm known as rotating calipers, however the implementations of this algorithm seem to rely on the idea that you have a set of vertices to begin with. Some details on this algorithm: https://www.geometrictools.com/Documentation/MinimumAreaRectangle.pdf
My main issue is in finding the vertices within the data that I currently have. I believe I need to at least find candidate vertices in order to reduce the amount of iterations I am performing, since the amount of points is relatively large and treating the interior points as if they are vertices is unnecessary if I can figure out a way to not include them.
Here is some example data that I am working with:
Here's the segmented scene using the naive algorithm, where it segments out the central objects relatively well due to the objects mostly being aligned with the image axes:
.
In red, you can see the current bounding boxes that I am drawing utilizing 2 vertices: top-left and bottom-right corners of the groups of points I have found.
The rotation part is where my current approach fails, as I am only defining the bounding box using two points, anything that is rotated and not axis-aligned will occupy much more area than necessary to encapsulate the points.
Here's an example with rotated objects in the scene:
Here's the current naive segmentation's performance on that scene, which is drawing larger than necessary boxes around the rotated objects:
Ideally the result would be bounding boxes aligned with the longest axis of the points that are being segmented, which is what I am having trouble implementing.
Here's an image roughly showing what I am really looking to accomplish:
You can also notice unnecessary segmentation done in the image around the borders as well as some small segments, which should be removed with some further heuristics that I have yet to develop. I would also be open to alternative segmentation algorithm suggestions that provide a more robust detection of the objects I am interested in.
I am not sure if this question will be completely clear, therefore I will try my best to clarify if it is not obvious what I am asking.
It's late, but that might still help. This is what you need to do:
expand pixels to make small segments connect larger bodies
find connected bodies
select a sample of pixels from each body
find the MBR ([oriented] minimum bounding rectangle) for selected set
For first step you can perform dilation. It's somehow like DBSCAN clustering. For step 3 you can simply select random pixels from a uniform distribution. Obviously the more pixels you keep, the more accurate the MBR will be. I tested this in MATLAB:
% import image as a matrix of 0s and 1s
oI = ~im2bw(rgb2gray(imread('vSb2r.png'))); % original image
% expand pixels
dI = imdilate(oI,strel('disk',4)); % dilated
% find connected bodies of pixels
CC = bwconncomp(dI);
L = labelmatrix(CC) .* uint8(oI); % labeled
% mark some random pixels
rI = rand(size(oI))<0.3;
sI = L.* uint8(rI) .* uint8(oI); % sampled
% find MBR for a set of connected pixels
for i=1:CC.NumObjects
[Y,X] = find(sI == i);
mbr(i) = getMBR( X, Y );
end
You can also remove some ineffective pixels using some more processing and morphological operations:
remove holes
find boundaries
find skeleton
In MATLAB:
I = imfill(I, 'holes');
I = bwmorph(I,'remove');
I = bwmorph(I,'skel');
What is Distance Transform?What is the theory behind it?if I have 2 similar images but in different positions, how does distance transform help in overlapping them?The results that distance transform function produce are like divided in the middle-is it to find the center of one image so that the other is overlapped just half way?I have looked into the documentation of opencv but it's still not clear.
Look at the picture below (you may want to increase you monitor brightness to see it better). The pictures shows the distance from the red contour depicted with pixel intensities, so in the middle of the image where the distance is maximum the intensities are highest. This is a manifestation of the distance transform. Here is an immediate application - a green shape is a so-called active contour or snake that moves according to the gradient of distances from the contour (and also follows some other constraints) curls around the red outline. Thus one application of distance transform is shape processing.
Another application is text recognition - one of the powerful cues for text is a stable width of a stroke. The distance transform run on segmented text can confirm this. A corresponding method is called stroke width transform (SWT)
As for aligning two rotated shapes, I am not sure how you can use DT. You can find a center of a shape to rotate the shape but you can also rotate it about any point as well. The difference will be just in translation which is irrelevant if you run matchTemplate to match them in correct orientation.
Perhaps if you upload your images it will be more clear what to do. In general you can match them as a whole or by features (which is more robust to various deformations or perspective distortions) or even using outlines/silhouettes if they there are only a few features. Finally you can figure out the orientation of your object (if it has a dominant orientation) by running PCA or fitting an ellipse (as rotated rectangle).
cv::RotatedRect rect = cv::fitEllipse(points2D);
float angle_to_rotate = rect.angle;
The distance transform is an operation that works on a single binary image that fundamentally seeks to measure a value from every empty point (zero pixel) to the nearest boundary point (non-zero pixel).
An example is provided here and here.
The measurement can be based on various definitions, calculated discretely or precisely: e.g. Euclidean, Manhattan, or Chessboard. Indeed, the parameters in the OpenCV implementation allow some of these, and control their accuracy via the mask size.
The function can return the output measurement image (floating point) - as well as a labelled connected components image (a Voronoi diagram). There is an example of it in operation here.
I see from another question you have asked recently you are looking to register two images together. I don't think the distance transform is really what you are looking for here. If you are looking to align a set of points I would instead suggest you look at techniques like Procrustes, Iterative Closest Point, or Ransac.
Is it valid to use OpenCV fitEllipse for circle fitting.
fitEllipse() returns cv::RotatedRect how about averaging width and height to get fitted circle radius?
I think that the "validity" of using cv::fitEllipse for fitting circles depends on the precision you require for the fitting.
For example you can run your algorithm on a test set, fitting points with cv::fitEllipse and logging the length of the two axes of the ellipse, then have a look at the distributions of the ratio of two axes or at the difference between the major and the minor axis; you can find how much your supposed circles differ from a circle and then asses if you can use the cv::fitEllipse.
You can take the average of the width and the height of the cv::RotatedRect returned by cv::fitEllipse to get an approximation of the diameter of the circle (you wrote the radius but I think it was a trivial error).
You can have a look at this very readable article
UMBACH, Dale; JONES, Kerry N. A few methods for fitting circles to data. Instrumentation and Measurement, IEEE Transactions on, 2003, 52.6: 1881-1885. and write your own circle interpolator.
If you want to minimize the geometric error (the sum of the squares of the distances from the points to the circle, as explained in the Introduction of the article) you maybe need a reliable implementation of a non linear minimization algorithm.
Otherwise you can write a simple circle interpolator with the formulae from (II.8) to (II.15) (a closed-form solution wich minimize an error different from the geometric one) with some warning:
from an implementation point of view you have to take care of the usually warnings about roundoff error and truncation error.
the closed form solution cannot be robust enough in case of outlier points, in that case you may need to implement a robust interpolator like RANSAC (random choose three points, interpolate a circle with that three points with formulae from (25) to (34) from Weisstein, Eric W. "Circle." From MathWorld--A Wolfram Web Resource, compute the consensus and iterate). This warning applies also to the circle found with the minimization of the geometric error.
There is a function for circle fitting: minEnclosingCircle
I have to detect the pattern of 6 circles using opencv. I have detected the circles and their centroids by using thresholding and contour function in opencv.
Now I have to define the relation between these circles in a way that should be invariant to scale and rotation. With this I would be able to detect this pattern in various views. I have to use this pattern for determining the object pose.
How can I achieve scale/rotation invariance? Do you have any reference I could read about it?
To make your pattern invariant toward rotation & scale, you have to normalize the direction and the scale when detecting your pattern. Here is a simple algorithm to achieve this
detect centers and circle size (you say you have already achieved this - good!)
compute the average center using a simple mean. Express all the centers from this mean
find the farthest center using a simple norm (euclidian is good enough)
scale the center position and the circle sizes so that this maximum distance is 1.0
rotate the centers so that coordinates of the farthest one is (1.0, 0)
you're done. You are now the proud owner of a scale/rotation invariant pattern detector!! Congratulations!
Now you can find patterns, transform them as suggested, and compare center position & circle sizes.
It is not entirely clear to me if you need to find the rotation, or merely get rid of it, or detect if the circles actually form the pattern you linked. Either way, the answer is much the same.
I would start by finding the two circles that have only one neighbour. For each circle centroid calculate the distance to the closest two neighbours. If the distances differ in more than say 10%, the centroid belongs to an "end" circle (one of the top ones in your link).
Now that you have found the two end circles, rotate them so that they are horizontal to each other. If the other centroids are now above them, rotate another 180 degrees so that the pattern ends up in the orientation you want.
Now you can calculate the scaling from the average inter-centroid distance.
Hope that helps.
Your question sounds exactly like what the SURF algorithm does. It finds groups of interest and groups them together in a way invarant to rotation and scale, and can find the same object in other pictures.
Just search for OpenCV and SURF.
I am looking for the right set of algorithms to solve this image processing problem:
I have a distorted binary image containing a distorted rectangle
I need to find a good approximation of the 4 corner points of this rectangle
I can calculate the contour using OpenCV, but as the image is distorted it will often contain more than 4 corner points.
Is there a good approximation algorithm (preferably using OpenCV operations) to find the rectangle corner points using the binary image or the contour description?
The image looks like this:
Thanks!
Dennis
Use cvApproxPoly function to eliminate number of nodes of your contour, then filter out those contours that have too many nodes or have angles which much differ from 90 degrees. See also similar answer
little different answer, see
http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html
Look at the opencv function ApproxPoly. It approximates a polygon from a contour.
Try Harris Corner Detector. There is example in OpenCV package. You need to play with params for your image.
And see other OpenCV algorithms: http://www.comp.leeds.ac.uk/vision/opencv/opencvref_cv.html#cv_imgproc_features
I would try generalised Hough Transform it is a bit slow but deals well with distorted/incomplete shapes.
http://en.wikipedia.org/wiki/Hough_transform
This will work even if you start with some defects, i.e. your approxPolly call returns pent/hexagons. It will reduce any contour, transContours in example, to a quad, or whatever poly you wish.
vector<Point> cardPoly;// Quad storage
int PolyLines = 0;//PolyPoly counter ;)
double simplicity = 0.5;//Increment of adjustment, lower numbers may be more precise vs. high numbers being faster to cycle.
while(PolyLines != 4)//Adjust this
{
approxPolyDP(transContours, Poly, simplicity, true);
PolyLines = Poly.size();
simplicity += 0.5;
}