Multiple Face Detection - opencv

I have a code in OpenCV (in C++) which uses "haarcascade_mcs_upperbody.xml" to detect upper body.
It detects single upper body. How can I make it detect multiple upper bodies.
I think CV_HAAR_FIND_BIGGEST_OBJECT is detecting only the biggest object. But I am not knowing how to solve this issue
The code goes like this:
int main(int argc, const char** argv)
{
CascadeClassifier body_cascade;
body_cascade.load("haarcascade_mcs_upperbody.xml");
VideoCapture captureDevice;
captureDevice.open(0);
Mat captureFrame;
Mat grayscaleFrame;
namedWindow("outputCapture", 1);
//create a loop to capture and find faces
while(true)
{
//capture a new image frame
captureDevice>>captureFrame;
//convert captured image to gray scale and equalize
cvtColor(captureFrame, grayscaleFrame, CV_BGR2GRAY);
equalizeHist(grayscaleFrame, grayscaleFrame);
//create a vector array to store the face found
std::vector<Rect> bodies;
//find faces and store them in the vector array
body_cascade.detectMultiScale(grayscaleFrame, faces, 1.1, 3,
CV_HAAR_FIND_BIGGEST_OBJECT|CV_HAAR_SCALE_IMAGE, Size(30,30));
//draw a rectangle for all found faces in the vector array on the original image
for(int i = 0; i < faces.size(); i++)
{
Point pt1(bodies[i].x + bodies[i].width, bodies[i].y + bodies[i].height);
Point pt2(bodies[i].x, bodies[i].y);
rectangle(captureFrame, pt1, pt2, cvScalar(0, 255, 0, 0), 1, 8, 0);
}
//print the output
imshow("outputCapture", captureFrame);
//pause for 33ms
waitKey(33);
}
return 0;
}

It seems there is some inconsistency in your code, since face_cascade is not defined anywhere, but I assume its type is CascadeClassifier.
detectMultiScale stores all detected objects in the faces vector. Are you sure it contains only one object?
Try removing the CV_HAAR_FIND_BIGGEST_OBJECT flag, because you want all objects to be detected, and not only the biggest one.
Also, make sure you set the minSize and maxSize parameters correctly (see documentation), since those parameters determine the minimal and maximal detectable object sizes.

Related

Avoiding memory leaks while using vector<Mat>

I am trying to write a code that uses opencv Mat objects it goes something like this
Mat img;
vector<Mat> images;
for (i = 1; i < 5; i++)
{
img.create(h,w,type) // h,w and type are given correctly
// input an image from somewhere to img correctly.
images.push_back(img);
img.release()
}
for (i = 1; i < 5; i++)
images[i].release();
I however still seem to have memory leakage can anyone tell me why it is so?
I thought that if the refcount of a mat object = 0 then the memory should be automatically deallocated
You rarely need to call release explicitly, since OpenCV Mat objects take automatically care of internal memory.
Also take care that Mat copy just copies creates a new header pointing to the same data. If the original Mat goes out of scope you are left with an invalid matrix. So when you push the image into the vector, use a deep copy (clone()) to avoid that it the image into the vector becomes invalid.
Since you mentioned:
I have a large 3D image stored in a Mat object. I am running over it using for loops. creating a 2D mat called "image" putting the slices into image, pushing back image to vector images. releasing the image. And later doing a for loop on the images vector releasing all the matrices one by one.
You can store all slices into the vector with the following code. To release the images in the vector, just clear the vector.
#include <opencv2/opencv.hpp>
#include <vector>
using namespace cv;
using namespace std;
int main()
{
// Init the multidimensional image
int sizes[] = { 10, 7, 5 };
Mat data(3, sizes, CV_32F);
randu(data, Scalar(0, 0, 0), Scalar(1,1,1));
// Put slices into images
vector<Mat> images;
for (int z = 0; z < data.size[2]; ++z)
{
// Create the slice
Range ranges[] = { Range::all(), Range::all(), Range(z, z + 1) };
Mat slice(data(ranges).clone()); // with clone slice is continuous, but still 3d
Mat slice2d(2, &data.size[0], data.type(), slice.data); // make the slice a 2d image
// Clone the slice into the vector, or it becomes invalid when slice goes of of scope.
images.push_back(slice2d.clone());
}
// You can deallocate the multidimensional matrix now, if needed
data.release();
// Work with slices....
// Release the vector of slices
images.clear();
return 0;
}
Please try this code, which is basically what you do:
void testFunction()
{
// image width/height => 80MB images
int size = 5000;
cv::Mat img = cv::Mat(size, size, CV_8UC3);
std::vector<cv::Mat> images;
for (int i = 0; i < 5; i++)
{
// since image size is the same for i==0 as the initial image, no new data will be allocated in the first iteration.
img.create(size+i,size+i,img.type()); // h,w and type are given correctly
// input an image from somewhere to img correctly.
images.push_back(img);
// release the created image.
img.release();
}
// instead of manual releasing, a images.clear() would have been enough here.
for(int i = 0; i < images.size(); i++)
images[i].release();
images.clear();
}
int main()
{
cv::namedWindow("bla");
cv::waitKey(0);
for(unsigned int i=0; i<100; ++i)
{
testFunction();
std::cout << "another iteration finished" << std::endl;
cv::waitKey(0);
}
std::cout << "end of main" << std::endl;
cv::waitKey(0);
return 0;
}
After the first call of testFunction, memory will be "leaked" so that the application consumes 4 KB more memory on my device. But not more "leaks" after additional calls for me...
So this looks like your code is ok and the "memory leak" isn't related to that matrix creation and releasing, but maybe to some "global" things happening within the openCV library or C++ to optimize future function calls or memory allocations.
I encountered same problems when iterate openCV mat. The memory consumption can be 1.1G, then it stopped by warning that no memory. In my program, there are macro #define new new(FILE, LINE), crashed with some std lib. So I deleted all Overloading Operator about new/delete. When debugging, it has no error. But when it runs, I got "Debug Assertion Failed! Expression: _pFirstBlock == pHead". Following the instruction
Debug Assertion Error in OpenCV
I changed setting from MT (Release)/MTd (Debug)to
Project Properties >> Configuration Properties >> C/C++ >> Code Generation and changing the Runtime Library to:
Multi-threaded Debug DLL (/MDd), if you are building the Debug version of your code.
Multi-threaded DLL(/MD), if you are building the Release version of your code.
The memory leak is gone. The memory consumption is 38M.

How to recognize digits from the analog counter?

I'm trying to read the following kWh numbers from the counter. The problem is the tesseract OCR doesn't recognize the analog digits.
The question is: will it be a better idea to make the photos of all of the digits (from 0 to 9) at different positions (I mean when digit is in the center, when it is a little at the top and the number 2 is appearing etc.) and to try image recognition instead of text recognition?
As far as I understood the difference is, that the image recognition compares the photos, while the text recognition... well I don't know...
Any advice?
Since the counter is not digital, but analog, we have problems at the transitions. The text/number recognition libraries can not recognize smth like that. The solution, that I've found is: Machine Learning.
Firstly I've made user to make the picture, where the numbers take 70-80% of the image (in order to remove the unneeded details).
Then I'm looking for parallel lines (if there are any) and cut the picture, that is between them (if the distance is big enough).
After that I'm filtering the picture (playing with contrast, brightness, set it grayscale) and then use the filter, that makes the image to contain only two colours (#000000 (black) and #ffffff (white)). In order to find the contours easier.
Then I find the contours by using Canny algorithm and filter them, by removing the unneeded details.
After that I use K-Nearest-Neighbour algorithm in order to recognize the digits.
But before I can recognize anything, I need to teach the algorithm, how the digits look like and what are they.
I hope it was useful!
Maybe you are not configuring tesseract right. I made a code using it that solves your problem:
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <tesseract/baseapi.h>
#include <iostream>
using namespace cv;
int main(int argc, char** argv)
{
cv::Mat input = cv::imread("img.jpg");
//rectangle containing just the kWh numbers
Rect roi(358,327,532,89);
//convert to gray scale
Mat input_gray;
cvtColor(input(roi),input_gray,CV_BGR2GRAY);
//threshold image
Mat binary_img = input_gray>200;
//make a copy to use on findcontours
Mat copy_binary_img = binary_img.clone();
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
//identify each blob in order to eliminate the small ones
findContours(copy_binary_img, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0,0));
//filter blobs by their sizes
for (vector<vector<Point> >::iterator it = contours.begin(); it!=contours.end(); )
{
if (it->size()>20)
it=contours.erase(it);
else
++it;
}
//Erase blobs which have countour size smaller than 20
for( int i = 0; i< contours.size(); i++ )
{
drawContours( binary_img, contours, i, 0, -1, 8, hierarchy, 0, Point() );
}
//initialize tesseract OCR
tesseract::TessBaseAPI tess;
tess.Init(NULL, "eng", tesseract::OEM_DEFAULT);
tess.SetVariable("tessedit_char_whitelist", "0123456789-.");
tess.SetPageSegMode(tesseract::PSM_SINGLE_BLOCK);
//set input
tess.SetImage((uchar*)binary_img.data
, binary_img.cols
, binary_img.rows
, 1
, binary_img.cols);
// Get the text
char* out = tess.GetUTF8Text();
std::cout << out << std::endl;
waitKey();
return 0;
}

Real time circle detection using OpenCV

I have written the following program to detect a circle in real time. But it doesn't work.
The compiler doesn't show any error but the problem is that the program doesn't even detect a circle. How can I fix it?
here is my code
using namespace cv;
int main()
{
VideoCapture cap(0);
namedWindow("main",CV_WINDOW_AUTOSIZE);
namedWindow("blur",CV_WINDOW_AUTOSIZE);
Mat img;
Mat img2;
int c;
float radius;
while(1)
{
cap>>img;
imshow("main",img);
cvtColor(img,img2,CV_BGR2GRAY);
GaussianBlur(img2,img2,Size(9,9),2,2);
imshow("blur",img2);
vector <Vec3f> circles;
HoughCircles(img2,circles,CV_HOUGH_GRADIENT,1,img2.rows/8,200,100,0,0);
for(size_t i=0;i<circles.size();i++)
{
Point center(cvRound(circles[i][0]),cvRound(circles[i][1]));
radius = cvRound(circles[i][2]);
circle(img,center,3,Scalar(0,255,0),-1,8,0);
circle(img,center,radius,Scalar(0,0,255),3,8,0);
}
c = waitKey(33);
if(c==27)
break;
}
destroyAllWindows();
return 0;
}
I checked your program, it seems you just forgot to visualize it using imshow() after the detection. You only drew the image before the detection, in this way, you were not able to see the circles (maybe this mistakenly make you think there is no circles detected) even it did detect some circles.
Try to add
imshow("main", img);
right before c = waitKey(33);.
You will see the circles if it does detect some circles.
Edit: to answer your comment for real time circle detection:
Do it in a while loop style will make it work for video frames. However, whether it is real time or not depends on how fast HoughCircles() will work and also other stuff inside the loop despite you setup the proper time for waitKey().

How to detect human using findcontours based on the human shape?

I wanna ask how to detecting humans or pedestrians on blob (findcontours)? I've try to learn how to detecting any object on the frame using findcontours() like this:
#include"stdafx.h"
#include<vector>
#include<iostream>
#include<opencv2/opencv.hpp>
#include<opencv2/core/core.hpp>
#include<opencv2/imgproc/imgproc.hpp>
#include<opencv2/highgui/highgui.hpp>
int main(int argc, char *argv[])
{
cv::Mat frame;
cv::Mat fg;
cv::Mat blurred;
cv::Mat thresholded;
cv::Mat thresholded2;
cv::Mat result;
cv::Mat bgmodel;
cv::namedWindow("Frame");
cv::namedWindow("Background Model"
//,CV_WINDOW_NORMAL
);
//cv::resizeWindow("Background Model",400,300);
cv::namedWindow("Blob"
//,CV_WINDOW_NORMAL
);
//cv::resizeWindow("Blob",400,300);
cv::VideoCapture cap("campus3.avi");
cv::BackgroundSubtractorMOG2 bgs;
bgs.nmixtures = 3;
bgs.history = 1000;
bgs.varThresholdGen = 15;
bgs.bShadowDetection = true;
bgs.nShadowDetection = 0;
bgs.fTau = 0.5;
std::vector<std::vector<cv::Point>> contours;
for(;;)
{
cap >> frame;
cv::GaussianBlur(frame,blurred,cv::Size(3,3),0,0,cv::BORDER_DEFAULT);
bgs.operator()(blurred,fg);
bgs.getBackgroundImage(bgmodel);
cv::threshold(fg,thresholded,70.0f,255,CV_THRESH_BINARY);
cv::threshold(fg,thresholded2,70.0f,255,CV_THRESH_BINARY);
cv::Mat elementCLOSE(5,5,CV_8U,cv::Scalar(1));
cv::morphologyEx(thresholded,thresholded,cv::MORPH_CLOSE,elementCLOSE);
cv::morphologyEx(thresholded2,thresholded2,cv::MORPH_CLOSE,elementCLOSE);
cv::findContours(thresholded,contours,CV_RETR_CCOMP,CV_CHAIN_APPROX_SIMPLE);
cv::cvtColor(thresholded2,result,CV_GRAY2RGB);
int cmin = 50;
int cmax = 1000;
std::vector<std::vector<cv::Point>>::iterator itc=contours.begin();
while (itc!=contours.end()) {
if (itc->size() > cmin && itc->size() < cmax){
std::vector<cv::Point> pts = *itc;
cv::Mat pointsMatrix = cv::Mat(pts);
cv::Scalar color( 0, 255, 0 );
cv::Rect r0= cv::boundingRect(pointsMatrix);
cv::rectangle(frame,r0,color,2);
++itc;
}else{++itc;}
}
cv::imshow("Frame",frame);
cv::imshow("Background Model",bgmodel);
cv::imshow("Blob",result);
if(cv::waitKey(30) >= 0) break;
}
return 0;
}
and now I wanna know how to detect humans? am I need to use hog? or haar? if yes I need to use them, how to use them? any tutorials to learn how to use it? because I'm so curious! and it's so much fun when I learn OpenCV! so addictive! :))
anyway I'll appreciate any help here, thanks. :)
This is a good start, with lots of enthusiasm. There is more than one way to do human detection on images/image sequences. I summarize a few below:
Since you are already extracting blobs that are supposed to be persons or objects, you can compare the features of these blobs with those of blobs resulting from a human in the scene. Many people look at the shape of the head-shoulder region, the height and area of the blob, etc.
You can also look at research papers like this one. The earlier researches are easier to understand and code, compared to the recent papers.
Instead of using background subtraction, you can also use an approach like Haar Wavelet based detection. This is widely used for face detection, but opencv contains a model for upper body detection. You can also build your own models, as described here.
Have fun!

OpenCV C++/Obj-C: goodFeaturesToTrack inside specific blob

Is there a quick solution to specify the ROI only within the contours of the blob I'm intereseted in?
My ideas so far:
Using the boundingRect, but it contains too much stuff I don't want to analyse.
Applying goodFeaturesToTrack to the whole image and then loop through the output coordinates to eliminate the once outside my blobs contour
Thanks in advance!
EDIT
I found what I need: cv::pointPolygonTest() seems to be the right thing, but I'm not sure how to implement it …
Here's some code:
// ...
IplImage forground_ipl = result;
IplImage *labelImg = cvCreateImage(forground.size(), IPL_DEPTH_LABEL, 1);
CvBlobs blobs;
bool found = cvb::cvLabel(&forground_ipl, labelImg, blobs);
IplImage *imgOut = cvCreateImage(cvGetSize(&forground_ipl), IPL_DEPTH_8U, 3);
if (found) {
vb::CvBlob *greaterBlob = blobs[cvb::cvGreaterBlob(blobs)];
cvb::cvRenderBlob(labelImg, greaterBlob, &forground_ipl, imgOut);
CvContourPolygon *polygon = cvConvertChainCodesToPolygon(&greaterBlob->contour);
}
"polygon" contains the contour I need.
goodFeaturesToTrack is implemented this way:
- (std::vector<cv::Point2f>)pointsFromGoodFeaturesToTrack:(cv::Mat &)_image
{
std::vector<cv::Point2f> corners;
cv::goodFeaturesToTrack(_image,corners, 100, 0.01, 10);
return corners;
}
So next I need to loop through the corners and check each point with cv::pointPolygonTest(), right?
You can create a mask over your interest region:
EDIT
How to make a mask:
Make a mask;
Mat mask(origImg.size(), CV_8UC1);
mask.setTo(Scalar::all(0));
// here I assume your contour is extracted with findContours,
// and is stored in a vector<vector<Point>>
// and that you know which contour is the blob
// if it's not the case, use fillPoly instead of drawContour();
Scalar color(255,255,255); // white. actually, it's monchannel.
drawContours(mask, contours, contourIdx, color );
// fillPoly(Mat& img, const Point** pts, const int* npts,
// int ncontours, const Scalar& color)
And now you're ready to use it. BUT, look carefully at the result - I have heard about some bugs in OpenCV regarding the mask parameter for feature extractors, and I am not sure if it's about this one.
// note the mask parameter:
void goodFeaturesToTrack(InputArray image, OutputArray corners, int maxCorners,
double qualityLevel, double minDistance,
InputArray mask=noArray(), int blockSize=3,
bool useHarrisDetector=false, double k=0.04 )
This will also improve the speed of your aplication - goodFeaturesToTrack eats a hoge amount of time, and if you apply it only on a smaller image, the overall gain is significant.

Resources