I can detect rectangles that are separate from each other. However, I am having problems with rectangles in contact such as below:
Two rectangles in contact
I should detect 2 rectangles in the image. I am using findContours as expected and I have tried various modes:CV_RETR_TREE, CV_RETR_LIST. I always get the outermost single contour as shown below:
Outermost contour detected
I have tried with or without canny edge detection. What I do is below:
cv::Mat element = cv::getStructuringElement(cv::MORPH_RECT, cv::Size(3,3));
cv::erode(__mat,__mat, element);
cv::dilate(__mat,__mat, element);
// Find contours
std::vector<std::vector<cv::Point> > contours;
cv::Mat coloredMat;
cv::cvtColor(__mat, coloredMat, cv::COLOR_GRAY2BGR);
int thresh = 100;
cv::Mat canny_output;
cv::Canny( __mat, canny_output, thresh, thresh*2, 3 );
cv::findContours(canny_output, contours, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
How can I detect both two rectangles separately?
If you already know the dimensions of the rectangle, you can use generalizedHoughTransform
If the dimensions of the rectangles are not known, you can use distanceTransform. The local maxima will give you the center location as well as the distance from the center to the nearest edge (which will be equal to half the short side of your rect). Further processing with corner detection / watershed and you should be able to find the orientation and dimensions (though this method may fail if the two rectangles overlap each other by a lot)
simple corner detection and brute force search (just try out all possible rectangle combinations given the corner points and see which one best matches the image, note that a rectangle can be defined given only 3 points) might also work
Related
I'm trying to build an algorithm that calculates the dimensions of slabs (in pixel units as of now). I tried masking, but there is no one HSV color range that will work for all the test cases, as the slabs are of varying colors. I tried Otsu thresholding as well but it didn't work quite well...
Now I'm trying my hand with canny edge detection. The original image, and the image after canny-edge look like this:
I used dilation to make the central region a uniform white region, and then used contour detection. I identified the contour having the maximum area as the contour of interest. The resulting contours are a bit noisy, because the canny edge detection also included some background stuff that was irrelevant:
I used cv2.boundingRect() to estimate the height and width of the rectangle, but it keeps returning the height and width of the entire image. I presume this is because it works by calculating (max(x)-min(x),max(y)-min(y)) for each (x,y) in the contour, and in my case the resulting contour has some pixels touching the edges of the image, and so this calculation simply results in (image width, image height).
I am trying to get better images to work with, but assuming all images are like this only, i.e. have noisy contours, what can be an alternate approach to detect the dimensions of the white rectangular region obtained after dilating?
To get the right points of the rectangle use this:
p = cv2.arcLength(cnt True) # cnt is the rect Contours
appr = cv2.approxPolyDP(cnt , 0.01 * p, True) # appr contains the 4 points
# draw the rect
cv2.drawContours(img, [appr], 0, (0, 255, 0), 2)
The appr var contains the turning point of the rect. You still need to do some more cleaning to get better results, but cv2.boundingRect() is not a good solution for your case.
I need find edges of document that in user hands.
1) Original image from camera:
2) Then i convert image to BG:
3) Then i make blur:
3) Finds edges in an image using the Canny:
4) And use dilate :
As you can see on the last image the contour around the map is torn and the contour is not determined. What is my error and how to solve the problem in order to determine the outline of the document completely?
This is code how i to do it:
final Mat mat = new Mat();
sourceMat.copyTo(mat);
//convert the image to black and white
Imgproc.cvtColor(mat, mat, Imgproc.COLOR_BGR2GRAY);
//blur to enhance edge detection
Imgproc.GaussianBlur(mat, mat, new Size(5, 5), 0);
if (isClicked) saveImageFromMat(mat, "blur", "blur");
//convert the image to black and white does (8 bit)
int thresh = 128;
Imgproc.Canny(mat, mat, thresh, thresh * 2);
//dilate helps to connect nearby line segments
Imgproc.dilate(mat, mat,
Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(3, 3)),
new Point(-1, -1),
2,
1,
new Scalar(1));
This answer is based on my above comment. If someone is holding the document, you cannot see the edge that is behind the user's hand. So, any method for detecting the outline of the document must be robust to some missing parts of the edge.
I suggest using a variant of the Hough transform to detect the document. The Wikipedia article about the Hough transform makes it sound quite scary (as Wikipedia often does with mathematical subjects), but don't be discouraged, actually they are not too difficult to understand or implement.
The original Hough transform detected straight lines in images. As explained in this OpenCV tutorial, any straight line in an image can be defined by 2 parameters: an angle θ and a distance r of the line from the origin. So you quantize these 2 parameters, and create a 2D array with one cell for every possible line that could be present in your image. (The finer the quantization you use, the larger the array you will need, but the more accurate the position of the found lines will be.) Initialize the array to zeros. Then, for every pixel that is part of an edge detected by Canny, you determine every line (θ,r) that the pixel could be part of, and increment the corresponding bin. After processing all pixels, you will have, for each bin, a count of how many pixels were detected on the line corresponding to that bin. Counts which are high enough probably represent real lines in the image, even if parts of the line are missing. So you just scan through the bins to find bins which exceed the threshold.
OpenCV contains Hough detectors for straight lines and circles, but not for rectangles. You could either use the line detector and check for 4 lines that form the edges of your document; or you could write your own Hough detector for rectangles, perhaps using the paper Jung 2004 for inspiration. Rectangles have at least 5 degrees of freedom (2D position, scale, aspect ratio, and rotation angle), and memory requirement for a 5D array obviously goes up pretty fast. But since the range of each parameter is limited (ie, the document's aspect ratio is known, and you can assume the document will be well centered and not rotated much) it is probably feasible.
I am trying to detect two concentric circles using opencv in Android. Big outer circle is red, inner smaller circle is blue. The idea is to detect big circle while distance is long and detect inner circle as the distance becomes short.
Sample picture
I am using simple code:
Mat matRed = new Mat();
Core.inRange(matHsv, getScalar(hue - HUE_D, saturation - SAT_D, brightness - BRIGHT_D), getScalar(hue + HUE_D, saturation + SAT_D, brightness + BRIGHT_D), matRed);
//here we have black-white image
Imgproc.GaussianBlur(matRed, matRed, new Size(0, 0), 6, 6);
Mat matCircles = new Mat();
Imgproc.HoughCircles(matRed, matCircles, CV_HOUGH_GRADIENT, 1, matRed.rows()/8, 100, param2, 0, 0);
After calling inRange we have white ring on black background. HoughCircles function detects only inner black circle.
How can I make it to detect outer white circle instead?
Without seeing a sample image (or being quite sure what you mean by 'detect big circle while distance is long and detect inner circle as the distance becomes short'), this is somewhat of a guess, but I'd suggest using Canny edge detect to get the boundaries of your circles and then using contours to extract the edges. You can use the contour hierarchy to determine which is inside which if you need to extract one or the other.
Additionally, given the circles are different colours, you might want to look at using inRange to segment based on colour; for example, this post from PyImageSearch contains a Python application which does colour-based tracking.
I have the following image.
this image
I would like to remove the orange boxes/rectangle around numbers and keep the original image clean without any orange grid/rectangle.
Below is my current code but it does not remove it.
Mat mask = new Mat();
Mat src = new Mat();
src = Imgcodecs.imread("enveloppe.jpg",Imgcodecs.CV_LOAD_IMAGE_COLOR);
Imgproc.cvtColor(src, hsvMat, Imgproc.COLOR_BGR2HSV);
Scalar lowerThreshold = new Scalar(0, 50, 50);
Scalar upperThreshold = new Scalar(25, 255, 255);
Mat mask = new Mat();
Core.inRange(hsvMat, lowerThreshold, upperThreshold, mask);
//src.setTo(new scalar(255,255,255),mask);
what to do next ?
How can i remove the orange boxes/rectangle from the original images ?
Update:
For information , the mask contains exactly all the boxes/rectangle that i want to remove. I don't know how to use this mask to remove boxes/rectangle from the source (src) image as if they were not present.
This is what I did to solve the problem. I solved the problem in C++ and I used OpenCV.
Part 1: Find box candidates
Firstly I wanted to isolate the signal that was specific for red channel. I splitted the image into three channels. I then subtracted the red channel from blue channel and the red from green channel. After that I subtracted both previous subtraction results from one another. The final subtraction result is shown on the image below.
using namespace cv;
using namespace std;
Mat src_rgb = imread("image.jpg");
std::vector<Mat> channels;
split(src_rgb, channels);
Mat diff_rb, diff_rg;
subtract(channels[2], channels[0], diff_rb);
subtract(channels[2], channels[1], diff_rg);
Mat diff;
subtract(diff_rb, diff_rg, diff);
My next goal was to divide the parts of obtained image into separate "groups". To do that, I smoothed the image a little bit with a Gaussian filter. Then I applied a threshold to obtain a binary image; finally I looked for external contours within that image.
GaussianBlur(diff, diff, cv::Size(11, 11), 2.0, 2.0);
threshold(diff, diff, 5, 255, THRESH_BINARY);
vector<vector<Point>> contours;
findContours(diff, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
Click to see subtraction result, Gaussian blurred image, thresholded image and detected contours.
Part 2: Inspect box candidates
After that, I had to make an estimate whether the interior of each contour contained a number or something else. I made an assumption that numbers will always be printed with black ink and that they will have sharp edges. Therefore I took a blue channel image and I applied just a little bit of Gaussian smoothing and convolved it with a Laplacian operator.
Mat blurred_ch2;
GaussianBlur(channels[2], blurred_ch2, cv::Size(7, 7), 1, 1);
Mat laplace_result;
Laplacian(blurred_ch2, laplace_result, -1, 1);
I then took the resulting image and applied the following procedure for every contour separately. I computed a standard deviation of the pixel values within the contour interior. Standard deviation was high inside the contours that surrounded numbers; and it was low inside the two contours that surrounded the dog's head and the letters on top of the stamp.
That is why I could appliy the standard deviation threshold. Standard deviation was approx. twice larger for contours containing numbers so this was an easy way to only select the contours that contained numbers. Then I drew the contour interior mask. I used erosion and subtraction to obtain the "box edge mask".
The final step was fairly easy. I computed an estimate of average pixel value nearby the box on every channel of the image. Then I changed all pixel values under the "box edge mask" to those values on every channel. After I repeated that procedure for every box contour, I merged all three channels into one.
Mat mask(src_rgb.size(), CV_8UC1);
for (int i = 0; i < contours.size(); ++i)
{
mask.setTo(0);
drawContours(mask, contours, i, cv::Scalar(200), -1);
Scalar mean, stdev;
meanStdDev(laplace_result, mean, stdev, mask);
if (stdev.val[0] < 10.0) continue;
Mat eroded;
erode(mask, eroded, cv::Mat(), cv::Point(-1, -1), 6);
subtract(mask, eroded, mask);
for (int c = 0; c < src_rgb.channels(); ++c)
{
erode(mask, eroded, cv::Mat());
subtract(mask, eroded, eroded);
Scalar mean, stdev;
meanStdDev(channels[c], mean, stdev, eroded);
channels[c].setTo(mean, mask);
}
}
Mat final_result;
merge(channels, final_result);
imshow("Final Result", final_result);
Click to see red channel of the image, the result of convolution with Laplacian operator, drawn mask of the box edges and the final result.
Please note
This code is far from being optimal, especially the last loop does quite a lot of unnecessary work. But I think that in this case readability is more important (and the author of the question did not request an optimized solution anyway).
Looking towards more general solution
After I posted the initial reply, the author of the question noted that the digits can be of any color and their edges are not necessarily sharp. That means that above procedure can fail because of various reasons. I altered the input image so that it contains different kinds of numbers (click to see the image) and you can run my algorithm on this input and analyze what goes wrong.
The way I see it, one of these approaches is needed (or perhaps a mixture of both) to obtain a more "general" solution:
concentrate only on rectangle shape and color (confirm that the box candidate is really an orange box and remove it regardless of what is inside)
concentrate on numbers only (run a proper number detection algorithm inside the interior of every box candidate; if it contains a single number, remove the box)
I will give a trivial example of the first approach. If you can assume that orange box size will always be the same, just check the box size instead of standard deviation of the signal in the last loop of the algorithm:
Rect rect = boundingRect(contours[i]);
float area = rect.area();
if (area < 1000 || area > 1200) continue;
Warning: actual area of rectangles is around 600Px^2, but I took into account the Gaussian Blurring, which caused the contour to expand. Please also note that if you use this approach you no longer need to perform blurring or laplace operations on blue channel image.
You can also add other simple constraints to that condition; ratio between width and height is the first one that comes to my mind. Geometric properties can also be a good option (right angles, straight edges, convexness ...).
I have an OpenCV application fed from a webcam stream of an office interior (lot's of details) where I have to find an artificial marker. The marker is a black square on white background. I use Canny to find edges and cvFindContours for contouring, then approxPolyDP and co. for filtering and finding candidates, then use local histogram to filter further, bla bla bla...
This works more or less, but not exactly how I want. FindContours always returns a closed loop, even if Canny creates a non-closed line. I get a contour walking on both sides of the line forming a loop. For closed edges on the Canny image (my marker), I get 2 contours, one on the inside, and an other on the outside.
I have to problems with this operation:
I get 2 contours for each marker (not that serious)
the most trivial filtering is not usable (reject non-closed contours)
So my question: is it possible to get non-closed contours for non-closed Canny edges?
Or what is the standard way to solve the above 2 issues?
Canny is a very good tool, but I need a way convert the 2D b/w image, into something easily process-able. Something like connected components listing all pixels in walking order of the component. So I can filter for loops, and feed it into approxPolyDP.
Update: I missed some important detail: the marker can be in any orientation (it's not front facing the camera, no right angles), in fact what I'm doing is 3D orientation estimation, based on the 2D projection of the marker.
I found a clean and easy solution for the 2 issues in the question. The trick is enable 2 level hierarchy generation (in findCountours) and look for contours which have a parent. This will return the inner contour of closed Canny edges and nothing more. Non-closed edges are discarded automatically, and each marker will have a single contour.
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
findContours(CannyImage, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_NONE, Point(0,0) );
for (unsigned int i=0; i<contours.size(); i++)
if (hierarchy[i][3] >= 0) //has parent, inner (hole) contour of a closed edge (looks good)
drawContours(contourImage, contours, i, Scalar(255, 0, 0), 1, 8);
It also works the other way around, that is: look for contours which have a child (hierarchy[i][2] >= 0), but in my case the parent check yields better results.
I had the same problem with duplicate contours and even dilate and erode could not solve it:
Mat src=imread("E:\\test.bmp"),gry,bin,nor,dil,erd;
GaussianBlur( src, nor, Size(5,5),0 );
cvtColor(nor,gry,CV_BGR2GRAY);
Canny(gry,bin,100,150,5,true);
dilate(bin,dil,Mat());
erode(dil,erd,Mat());
Mat tmp=bin.clone();
vector<vector<Point>> conts;
vector<Vec4i> hier;
findContours(tmp,conts,hier,CV_RETR_TREE,CV_CHAIN_APPROX_SIMPLE);
This image (test.bmp) contains 3 contours but findContours returned 6!
I used threshold and problem solved:
Mat src=imread("E:\\test.bmp"),gry,bin,nor,dil,erd;
GaussianBlur( src, nor, Size(5,5),0 );
cvtColor(nor,gry,CV_BGR2GRAY);
threshold(gry,bin,0,255,THRESH_BINARY+THRESH_OTSU);
vector<vector<Point>> conts;
vector<Vec4i> hier;
findContours(bin,conts,hier,CV_RETR_TREE,CV_CHAIN_APPROX_SIMPLE);
Now it returns 4 contours which the 1st one is the image boundary(contour with index 0) and can be easily skipped.
This how I would do it
1. Canny for edge detection
2. Use houghtransform to detect the edges.
3. Detect the two edges that do an angle of 90.