find objects with bounding boxes - opencv

I would like to find bounding boxes for each object in the picture and after I found that, I crop out the bounding boxes and use it for the next steps.Here is the input picture after preprocessing.
I have a code for bounding box, but it just works well for 1 object. If there are 2 objects it sums up both and draw a bounding box around both of them. Here is the first output. The code for it is:
vector<vector<Point>> contours;
vector<Point> points;
findContours(erod, contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
for (size_t i = 0; i < contours.size(); i++) {
for (size_t j = 0; j < contours[i].size(); j++) {
Point p = contours[i][j];
points.push_back(p);
}
}
if (points.size() > 0) {
Rect brect = boundingRect(Mat(points).reshape(2));
cv::rectangle(erod, brect.tl(), brect.br(), Scalar(100,100,200), 2, CV_AA);
Mat ROI = frame(brect);
}
The secound thing I tried was using the code of the documentation of OpenCV. Here I changed CV_RETR_TREE in findContours to CV_RETR_EXTERNAL but I still get to many bounding boxes and I don't know how to crop out the boxes.
Thanks a lot!

Before finding contours you should do some morphological opening to clear all the noise and lines:
Mat morphKernelOpen = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new org.opencv.core.Size(20, 20));
Imgproc.morphologyEx(mat, mat, Imgproc.MORPH_OPEN, morphKernelOpen);
Result:
Also, there are some black spaces inside your objects, so to avoid finding contours in them, your findContours function should be under CV_RETR_EXTERNAL mode:
Imgproc.findContours(scharrThresh, scharrThreshContours, new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
In the end you'll have two contours and you can continue your boxes finding as you did previously
If you do not like soft edges around your objects you can do threshold function before morphological opening. Getting 100% accurate contours around your objects would be very hard or nearly impossible due to too much noise in the image. Also, if you can, next time if you ask put the result image you get after the actions you do, it will be easier to give you a proper answer.

Related

Matchingproblems when using OpenCVs matchShapes function

I´m trying to find a objekt in a larger Picture with the findContour/matchShape functions (the object can vary so it´s not possible to look after the color or something similar, Featuredetectors like SIFT also doesn´t work because the object could be symetric)
I have written following code:
Mat scene = imread...
Mat Template = imread...
Mat imagegray1, imagegray2, imageresult1, imageresult2;
int thresh=80;
double ans=0, result=0;
// Preprocess pictures
cvtColor(scene, imagegray1,CV_BGR2GRAY);
cvtColor(Template,imagegray2,CV_BGR2GRAY);
GaussianBlur(imagegray1,imagegray1, Size(5,5),2);
GaussianBlur(imagegray2,imagegray2, Size(5,5),2);
Canny(imagegray1, imageresult1,thresh, thresh*2);
Canny(imagegray2, imageresult2,thresh, thresh*2);
vector<vector <Point> > contours1;
vector<vector <Point> > contours2;
vector<Vec4i>hierarchy1, hierarchy2;
// Template
findContours(imageresult2,contours2,hierarchy2,CV_RETR_EXTERNAL,CV_CHAIN_APPROX_SIMPLE,cvPoint(0,0));
// Szene
findContours(imageresult1,contours1,hierarchy1,CV_RETR_EXTERNAL,CV_CHAIN_APPROX_SIMPLE,cvPoint(0,0));
imshow("template", Template);
double helper = INT_MAX;
int idx_i = 0, idx_j = 0;
// Match all contours with eachother
for(int i = 0; i < contours1.size(); i++)
{
for(int j = 0; j < contours2.size(); j++)
{
ans=matchShapes(contours1[i],contours2[j],CV_CONTOURS_MATCH_I1 ,0);
// find the best matching contour
if((ans < helper) )
{
idx_i = i;
helper = ans;
}
}
}
// draw the best contour
drawContours(scene, contours1, idx_i,
Scalar(255,255,0),3,8,hierarchy1,0,Point());
When I'm using a scene where only the Template is located in, i get a good matching result:
But when there are more objects in the pictures i have trouble detecting the object:
Hope someone can tell me whats the problem with the code i´m using. Thanks
You have a huge amount of contours in the second image (almost each letter).
As the matchShape checks for scale-invariant Hu-moments (http://docs.opencv.org/3.1.0/d3/dc0/group__imgproc__shape.html#gab001db45c1f1af6cbdbe64df04c4e944) also a very small contours may fit the shape you are looking for.
Furthermore, the original shape is not distinguished properly like can be seen when excluding all contours with an area smaller 50.
if(contourArea(contours1[i]) > 50)
drawContours(scene, contours1, i, Scalar(255, 255, 0), 1);
To say it with other words, there is no problem with your code. The contour can simply not be detected very well. I would suggest to have a look at approxCurve and convexHull and try to close the contour this way. Or improve the use of Canny in some way.
Then you could use a priori knowledge to restrict the size (and maybe rotation?) of the contour you are looking for.

Too many points have been found in contours detected by Opencv findContours function

thanks for paying attention to this question.
I want to detect some moving objects by using Kinect Sensor. The idea is quite simple, which first I will get the difference image between every two frames, and then extract the contours of objects, finally do further processing.
I tried to extract the contours by using Opencv(version 2.4.9) function findContours, but here the problem comes. The function can extract about 30 or 40 contours in each loop, but in each contour there are about billions of points contained in the contours. Also, if I want to use some functions like drawContours or minAreaRect, the program will crash due to memory error.
Here are relative code:
findContours(Black, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE,
Point(0, 0));
//1. If nothing has entered the camera frame, skip
if(contours.size()==0)
{
//cout<<"NoContours ";
continue;
}
//2. Only save the maximum contour(index) as the result
max_index = 0;
for (size_t i = 1; i < contours.size(); i++)
{
//cout << " Num of Points: " << contours[i].size() << endl;
if(contours[max_index].size() < contours[i].size())
{
max_index = int(i);
}
}
//3. If the maximum contour's size is smaller than 5, regard it as noise
//cout << contours[max_index].size() << endl;
if(contours[max_index].size() < 5)
{
continue;
}
//find a smallest RotatedRect to express the contour(error happen)
minRect = minAreaRect(Mat(contours[max_index]));
RotatedRect minEllipse = fitEllipse(contours[max_index]);
Error will happen when it runs to the last two lines code. The mainly reason I think that the function findContours found too many points in every contours, which leads to not enough memory.
I cannot send an image for now, but the function findContours found about 4294966890 points in at least 50% of contours (while others are normal)
Could anyone give some idea about this?
Try to use approxpolydp to simplify your contours.

Segmentation of perspectively distorted barcodes

There are images with perspectively distorted barcodes in them.
They are located and decoded using ZBar.
Now I do not only need the rough location, but the four real corner points of the barcode, that define the enclosing 4-point polygon.
I tried different approaches, but did not yet get the desired result.
One of them was:
convert image to grayscale
threshold image
erode image
floodFill beginning with a pixel known to be part of barcode
obtain the contour around the floodFill result
But around this contour I now would need to find the minimum best fitting 4-point polygon, which seems to be not that easy.
Do you have ideas for better approaches?
You could use the following code and try to reduce your contour to 4-point polygon via approxPoly
vector approx;
for (size_t i = 0; i < contours.size(); i++)
{
approxPolyDP(Mat(contours[i]), approx,
arcLength(Mat(contours[i]), true)*0.02, true);
if (approx.size() == 4 &&
fabs(contourArea(Mat(approx))) > 1000 &&
isContourConvex(Mat(approx)))
{
double maxCosine = 0;
for( int j = 2; j < 5; j++ )
{
double cosine = fabs(angle(approx[j%4], approx[j-2], approx[j-1]));
maxCosine = MAX(maxCosine, cosine);
}
if( maxCosine < 0.3 )
squares.push_back(approx);
}
}
http://opencv-code.com/tutorials/detecting-simple-shapes-in-an-image/
You can also try the following methods, maybe they will produce good enough results for you:
http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=minarearect#minarearect
or
http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=convexhull#convexhull
OK, I found a solution that works good enough for my use case.
First a scanline is generated from the ZBar result.
Now the first and the last black pixels are found in verion of the image resulting from cv::adaptivethreshold with a large enough blockSize.
From there on the first and the last bar are segmented using cv::findContours.
Now for both end bars the two contour points with the most distance to each others are searched.
They finally define the enclosing 4-point-polygon.
Which is not exactly what I posted in my question, but the additional size due to the elongated guard patterns does not matter in my case.

How to track multiple object location?

I need to track multiple objects, some color objects, attached on human body; All same color. I can track one object through Threshold image and Moment but when I use more than one object the computed Moment is something between those two or three. I need to have xy coordinate of each ones. Actually, after all, I want to do some analysis on those sequences of coordinates.
I'm using VS2010, OpenCV 2.3.1, Win7 x64.
You have to compute the moments for each blob alone. To accomplish this, you can use cv::findContours to get a descriptor for each blob in the form of its contour, then use it to compute its moments. In the code snippet below, inspired by this example, it is shown how one would compute the mass centers of each blob using this approach.
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
// Find contours
cv::findContours(img, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
// Get the moments
std::vector<Moments> mu(contours.size() );
for(int i = 0; i < contours.size(); i++)
mu[i] = moments(contours[i], false);
// Get the mass centers:
std::vector<cv::Point2f> mc(contours.size());
for(int i = 0; i < contours.size(); i++)
mc[i] = Point2f(mu[i].m10/mu[i].m00 , mu[i].m01/mu[i].m00);

detect blob over other blob

I use OpenCV and cvblob library to play with blob.
Now I want to detect blob in this particular case.
The problem or the difficulty in this case is there are two blobs over a bigger one and other blob that overlap a part of the bigger one.
In cvblob library to detect a blob you must have a binary image.
I think i need to create two or more image to segment color uniform blobs and then binarize them to obtain all the blobs in the image.
How can i do that.
thanks in advance
I'm quite a beginner in OpenCV but I guess that, for that particular case, you should work with cvFindContours with the CV_RETR_EXTERNAL flag (with the CV_RETR_TREE, your yellow blob would be IN the blue one) instead of using cvblob.
It depends if you want to track them or not (cvblob offers a quick and efficient way to track blobs, instead of having to implement camshift).
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* firstContour = cvCreateSeq(CV_SEQ_ELTYPE_POINT, sizeof(CvSeq), sizeof(CvPoint), storage);
cvFindContours(image, storage, &firstContour, sizeof(CvContour), CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
// S'il y a un contour
if(firstContour != 0) {
for( CvSeq* c = firstContour; c != NULL; c = c->h_next ) {
for(int i = 0; i < c->total; ++i) {
// Get each point of the current contour
CvPoint* pt = CV_GET_SEQ_ELEM(CvPoint, c, i);
double x = pt->x;
double y = pt->y;
}
}
}
With the information given by the contour you can find easily the centroid, angle and bounding box of your blob.
Tracking these blob might be more difficult as cvblob doesn't like overlapping blobs (as I can see). You may have to implement your own tracking method.

Resources