Related
I want to find the contours of a binary image of segmented rocks. There are some problems with the findContours function from opencv.
The contour size is around 1000 while the contours from the binary image could be around 30-50.
When I draw ALL the contours, they seem to be a decent representation of the black boundaries from the binary image. But When I draw only one contour of some random index, it shows a small contour.
Images are given below :
Binary Image
Contours of all the index
Contour of a random contour index. The small green contour
I would like to have just the exact number of contours as in the binary image.
Code :
std::vector<std::vector<cv::Point>> contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(input_image, contours,hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_NONE);
for( int i = 0; i < (int)contours.size(); i++)
{
cv::drawContours(input_rgb_image, contours, 512 , cv::Scalar(0,255,0), 1, 8, hierarchy,1);
}
There are two problems with your code. You will get better results if you invert and blur the image. These are my results after applying those two operations before finding the contours:
The OpenCV findContours() function finds dark contours on the light background. If you want to find the white spaces, which are the rocks, you need to invert the binary image first. You can invert a binary image like this invertedImage = 255 - binaryImage. Blurring also helps because it connects pixels that should be connected but aren't because of the low resolution. Blurring is done with the code blurredImage = cv2.blur(img, (2,2)). This is the inverted blurred image:
This is the code that I used:
import cv2
import random
# Read image
gray = 255-cv2.imread('/home/stephen/Desktop/image.png', 0)
gray = cv2.blur(gray, (2,2))
# Find contours in image
contours, _ = cv2.findContours(gray, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
print(len(contours))
img = cv2.imread('/home/stephen/Desktop/image.png')
for cnt in contours:
color = random.randint(0,255),random.randint(0,255),random.randint(0,255)
img = cv2.drawContours(img, [cnt], 0, color, cv2.FILLED)
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
I would try a couple of things:
bilateral filter instead of blur. It smoothes things in a way similar
to blur but also tries to preserve boundaries, which is good for segmentation. Downsides - it's computationally expensive but you may
find "your" params that play well for free
blur + meanshift segmentation before the watershed. Blur will act just
like expected and meanshift will average and join contours with
similar colors and as such make the number of contours smaller.
Depending on params, meanshift is also expensive. Just play with
it.
More advanced thing is contours analysis afterward. You could unite some of the neighbors based on:
the similarity of the histogram on some of hsv channels;
contours properties, such as roundness. If roundness of two united
neighbors is better than the roundness of any of them then they can be united. Something like this.
Roundness calculating:
float calcRoundness(std::vector<cv::Point> &contour, double area)
{
float p = cv::arcLength(contour, true);
if (p == 0)
return 0;
float k = (4 * M_PI * area) / pow(p, 2);
/* 1 is circle, 0.75 - squared area, etc. */
return k;
}
I am trying to detect the mouth in an image with openCV, so I am using the following code:
#include "face_detection.h"
using namespace cv;
// Function detectAndDisplay
void detectAndDisplay(const std::string& file_name, cv::CascadeClassifier& face_cascade, cv::CascadeClassifier& mouth_cascade)
{
Mat frame = imread(file_name);
std::vector<Rect> faces;
Mat frame_gray;
Mat crop;
Mat res;
Mat gray;
cvtColor(frame, frame_gray, COLOR_BGR2GRAY);
equalizeHist(frame_gray, frame_gray);
// Detect faces
face_cascade.detectMultiScale(frame_gray, faces, 1.1, 3, 0 | CASCADE_SCALE_IMAGE, Size(30, 30));
for(unsigned int i=0;i<faces.size();i++)
{
rectangle(frame,faces[i],Scalar(255,0,0),1,8,0);
Mat face = frame(faces[i]);
cvtColor(face,face,CV_BGR2GRAY);
std::vector <Rect> mouthi;
mouth_cascade.detectMultiScale(face, mouthi);
for(unsigned int k=0;k<mouthi.size();k++)
{
Point pt1(mouthi[k].x+faces[i].x , mouthi[k].y+faces[i].y);
Point pt2(pt1.x+mouthi[k].width, pt1.y+mouthi[k].height);
rectangle(frame, pt1,pt2,Scalar(0,255,0),1,8,0);
}
}
imshow("Frame", frame);
waitKey(33);
}
The classifiers are haarcascade_frontalface_alt.xml and haarcascade_mcs_mouth.xml.
The face is detected correctly but the mouth is not: I also obtain the eyes and some other parts, like the forehead.
Is there a way to detect only the mouth?
I think I managed to solve the problem: focusing on the lower half of the face and increasing the scale factor did the trick and now I am able to detect the mouth with a good precision. Anyway this task seems much more complicated than face detection, even if I am using "simple" images, which means straight and full frontal.
Here are two examples: a success and a failure.
I was facing the same problem, so I focused only on the lower half of the face
and created an ROI from the detected face. It looks something like this:
Mat ROI=image(Rect(face.x,face.y+face.height*0.6,face.width,face.height*0.3));
Where face is the detected face from the image.
This created an ROI from the detected face for the lower half only. Else the mouth detector was detecting the eyes also as mouth.
Then use the MouthCascade.xml from this link: http://alereimondo.no-ip.org/OpenCV/34
which is far more efficient than the inbuilt OpenCV one.
I have a set of discrete points shown in an image, like the following
I want to reconstruct or up sampling (I'm not sure what's the correct way to describe it) the image, so that the result image would be like the following. It doesn't need to be exactly the same as the example image, but the main idea is to fill up the original one.
I have an initial idea about how to do it. But I don't know how to do it after the first step. My idea is to first separate image using kmeans and find out the different objects. And I have successfully done it. The resulting images after kmeans are: .
After kmeans, I want to use find contour or something like concave to get the outline of these shapes and fill the shape using functions like fill holes. However, I found "find contour" does not work, it will consider each single pixel as a contour.
The other way I'm thinking is to use interpolation. But I'm not sure whether it is possible with so sparse points. Does anyone have any ideas about how to do this? I'm not sure whether I'm on the right way and I'm open to any solutions.
Thanks a lot!
Take a look at the morphological transformations. I would start with a dilation operation using a large kernel, say the MORPH_ELLIPSE with a size(15,15). Afterwards, thin the blobs back down using the erosion operation with the same size kernel. Take a look at the docs here. Note that OpenCV offers chained, or sequenced, morphological operations, too. See here. You'll then see that my suggestion is a "closing" operation.
Update:
I experimented with simple dilation and contouring to yield the results shown in the image. The results appear to satisfy the general requirements of the problem.
Likewise, what "realtime" means for the application isn't specified, but this set of operations may be quickly executed and could easily be applied to a 30fps application.
Code snippet below:
// Convert image to grayscale
cvtColor(src, gray, CV_BGR2GRAY);
threshold(gray, gray, 128.0, 128.0, THRESH_BINARY);
// Dilate to fill holes
dilate(gray, dest, getStructuringElement(MORPH_ELLIPSE, Size(13,13)));
// Find contours
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
findContours(dest, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0,0));
// Prune contours
float maxArea = 0.0f;
for (size_t i = 0; i< contours.size(); i++)
{
if (contourArea(contours[i]) >= maxArea)
{
maxArea = contourArea(contours[i]);
}
}
float minArea = 0.20f * maxArea;
vector<vector<Point> > prunedContours;
for (size_t i = 0; i< contours.size(); i++)
{
if (contourArea(contours[i]) >= minArea)
{
prunedContours.push_back(contours[i]);
}
}
// Smooth the contours
vector<vector<Point> > smoothedContours;
smoothedContours.resize(prunedContours.size());
for (size_t i=0;i<prunedContours.size();i++)
{
vector<float> x;
vector<float> y;
const size_t n = prunedContours[i].size();
for (size_t j=0;j<n;j++)
{
x.push_back(prunedContours[i][j].x);
y.push_back(prunedContours[i][j].y);
}
Mat G;
transpose(getGaussianKernel(11,4.0,CV_32FC1),G);
vector<float> xSmooth;
vector<float> ySmooth;
filter2D(x,xSmooth, CV_32FC1, G);
filter2D(y,ySmooth, CV_32FC1, G);
for (size_t j=0;j<n;j++)
{
smoothedContours[i].push_back(Point2f(xSmooth[j],ySmooth[j]));
}
}
I'm currently working on Image stitching using OpenCV 2.3.1 on Visual Studio 2010, but I'm having some trouble.
Problem Description
I'm trying to write a code for stitching multiple images derived from a few cameras(about 3~4), i,e, the code should keep executing image stitching until I ask it to stop.
The following is what I've done so far:
(For simplification, I'll replace some part of the code with just a few words)
1.Reading frames(images) from 2 cameras (Currently I'm just working on 2 cameras.)
2.Feature detection, descriptor calculation (SURF)
3.Feature matching using FlannBasedMatcher
4.Removing outliers and calculate the Homography with inliers using RANSAC.
5.Warp one of both images.
For step 5., I followed the answer in the following thread and just changed some parameters:
Stitching 2 images in opencv
However, the result is terrible though.
I just uploaded the result onto youtube and of course only those who have the link will be able to see it.
http://youtu.be/Oy5z_7LeaMk
My code is shown below:
(Only crucial parts are shown)
VideoCapture cam1, cam2;
cam1.open(0);
cam2.open(1);
while(1)
{
Mat frm1, frm2;
cam1 >> frm1;
cam2 >> frm2;
//(SURF detection, descriptor calculation
//and matching using FlannBasedMatcher)
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_1.rows; i++ )
{
double dist = matches[i].distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
(Draw only "good" matches
(i.e. whose distance is less than 3*min_dist ))
vector<Point2f> frame1;
vector<Point2f> frame2;
for( int i = 0; i < good_matches.size(); i++ )
{
//-- Get the keypoints from the good matches
frame1.push_back( keypoints_1[ good_matches[i].queryIdx ].pt );
frame2.push_back( keypoints_2[ good_matches[i].trainIdx ].pt );
}
Mat H = findHomography( Mat(frame1), Mat(frame2), CV_RANSAC );
cout << "Homography: " << H << endl;
/* warp the image */
Mat warpImage2;
warpPerspective(frm2, warpImage2,
H, Size(frm2.cols, frm2.rows), INTER_CUBIC);
Mat final(Size(frm2.cols*3 + frm1.cols, frm2.rows),CV_8UC3);
Mat roi1(final, Rect(frm1.cols, 0, frm1.cols, frm1.rows));
Mat roi2(final, Rect(2*frm1.cols, 0, frm2.cols, frm2.rows));
warpImage2.copyTo(roi2);
frm1.copyTo(roi1);
imshow("final", final);
What else should I do to make the stitching better?
Besides, is it reasonable to make the Homography matrix fixed instead of keeping computing it ?
What I mean is to specify the angle and the displacement between the 2 cameras by myself so as to derive a Homography matrix that satisfies what I want.
Thanks. :)
It sounds like you are going about this sensibly, but if you have access to both of the cameras, and they will remain stationary with respect to each other, then calibrating offline, and simply applying the transformation online will make your application more efficient.
One point to note is, you say you are using the findHomography function from OpenCV. From the documentation, this function:
Finds a perspective transformation between two planes.
However, your points are not restricted to a specific plane as they are imaging a 3D scene. If you wanted to calibrate offline, you could image a chessboard with both cameras, and the detected corners could be used in this function.
Alternatively, you may like to investigate the Fundamental matrix, which can be calculated with a similar function. This matrix describes the relative position of the cameras, but some work (and a good textbook) will be required to extract them.
If you can find it, I would strongly recommend having a look at Part II: "Two-View Geometry" in the book "Multiple View Geometry in computer vision", by Richard Hartley and Andrew Zisserman, which goes through the process in detail.
I have been working lately on image registration. My algorithm takes two images, calculates the SURF features, find correspondences, find homography matrix and then stitch both images together, I did it with the next code:
void stich(Mat base, Mat target,Mat homography, Mat& panorama){
Mat corners1(1, 4,CV_32F);
Mat corners2(1,4,CV_32F);
Mat corners(1,4,CV_32F);
vector<Mat> planes;
/* compute corners
of warped image
*/
corners1.at<float>(0,0)=0;
corners2.at<float>(0,0)=0;
corners1.at<float>(0,1)=0;
corners2.at<float>(0,1)=target.rows;
corners1.at<float>(0,2)=target.cols;
corners2.at<float>(0,2)=0;
corners1.at<float>(0,3)=target.cols;
corners2.at<float>(0,3)=target.rows;
planes.push_back(corners1);
planes.push_back(corners2);
merge(planes,corners);
perspectiveTransform(corners, corners, homography);
/* compute size of resulting
image and allocate memory
*/
double x_start = min( min( (double)corners.at<Vec2f>(0,0)[0], (double)corners.at<Vec2f> (0,1)[0]),0.0);
double x_end = max( max( (double)corners.at<Vec2f>(0,2)[0], (double)corners.at<Vec2f>(0,3)[0]), (double)base.cols);
double y_start = min( min( (double)corners.at<Vec2f>(0,0)[1], (double)corners.at<Vec2f>(0,2)[1]), 0.0);
double y_end = max( max( (double)corners.at<Vec2f>(0,1)[1], (double)corners.at<Vec2f>(0,3)[1]), (double)base.rows);
/*Creating image
with same channels, depth
as target
and proper size
*/
panorama.create(Size(x_end - x_start + 1, y_end - y_start + 1), target.depth());
planes.clear();
/*Planes should
have same n.channels
as target
*/
for (int i=0;i<target.channels();i++){
planes.push_back(panorama);
}
merge(planes,panorama);
// create translation matrix in order to copy both images to correct places
Mat T;
T=Mat::zeros(3,3,CV_64F);
T.at<double>(0,0)=1;
T.at<double>(1,1)=1;
T.at<double>(2,2)=1;
T.at<double>(0,2)=-x_start;
T.at<double>(1,2)=-y_start;
// copy base image to correct position within output image
warpPerspective(base, panorama, T,panorama.size(),INTER_LINEAR| CV_WARP_FILL_OUTLIERS);
// change homography to take necessary translation into account
gemm(T, homography,1,T,0,T);
// warp second image and copy it to output image
warpPerspective(target,panorama, T, panorama.size(),INTER_LINEAR);
//tidy
corners.release();
T.release();
}
Any question I will try
I have a problem with filling white holes inside a black coin so that I can have only 0-255 binary images with filled black coins. I have used a Median filter to accomplish it but in that case connection bridge between coins grows and it goes impossible to recognize them after several times of erosion... So I need a simple floodFill like method in opencv
Here is my image with holes:
EDIT: floodfill like function must fill holes in big components without prompting X, Y coordinates as a seed...
EDIT: I tried to use the cvDrawContours function but it doesn't fill contours inside bigger ones.
Here is my code:
CvMemStorage mem = cvCreateMemStorage(0);
CvSeq contours = new CvSeq();
CvSeq ptr = new CvSeq();
int sizeofCvContour = Loader.sizeof(CvContour.class);
cvThreshold(gray, gray, 150, 255, CV_THRESH_BINARY_INV);
int numOfContours = cvFindContours(gray, mem, contours, sizeofCvContour, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
System.out.println("The num of contours: "+numOfContours); //prints 87, ok
Random rand = new Random();
for (ptr = contours; ptr != null; ptr = ptr.h_next()) {
Color randomColor = new Color(rand.nextFloat(), rand.nextFloat(), rand.nextFloat());
CvScalar color = CV_RGB( randomColor.getRed(), randomColor.getGreen(), randomColor.getBlue());
cvDrawContours(gray, ptr, color, color, -1, CV_FILLED, 8);
}
CanvasFrame canvas6 = new CanvasFrame("drawContours");
canvas6.showImage(gray);
Result: (you can see black holes inside each coin)
There are two methods to do this:
1) Contour Filling:
First, invert the image, find contours in the image, fill it with black and invert back.
des = cv2.bitwise_not(gray)
contour,hier = cv2.findContours(des,cv2.RETR_CCOMP,cv2.CHAIN_APPROX_SIMPLE)
for cnt in contour:
cv2.drawContours(des,[cnt],0,255,-1)
gray = cv2.bitwise_not(des)
Resulting image:
2) Image Opening:
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
res = cv2.morphologyEx(gray,cv2.MORPH_OPEN,kernel)
The resulting image is as follows:
You can see, there is not much difference in both cases.
NB: gray - grayscale image, All codes are in OpenCV-Python
Reference. OpenCV Morphological Transformations
A simple dilate and erode would close the gaps fairly well, I imagine. I think maybe this is what you're looking for.
A more robust solution would be to do an edge detect on the whole image, and then a hough transform for circles. A quick google shows there are code samples available in various languages for size invariant detection of circles using a hough transform, so hopefully that will give you something to go on.
The benefit of using the hough transform is that the algorithm will actually give you an estimate of the size and location of every circle, so you can rebuild an ideal image based on that model. It should also be very robust to overlap, especially considering the quality of the input image here (i.e. less worry about false positives, so can lower the threshold for results).
You might be looking for the Fillhole transformation, an application of morphological image reconstruction.
This transformation will fill the holes in your coins, even though at the cost of also filling all holes between groups of adjacent coins. The Hough space or opening-based solutions suggested by the other posters will probably give you better high-level recognition results.
In case someone is looking for the cpp implementation -
std::vector<std::vector<cv::Point> > contours_vector;
cv::findContours(input_image, contours_vector, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
cv::Mat contourImage(input_image.size(), CV_8UC1, cv::Scalar(0));
for ( ushort contour_index = 0; contour_index < contours_vector.size(); contour_index++) {
cv::drawContours(contourImage, contours_vector, contour_index, cv::Scalar(255), -1);
}
cv::imshow("con", contourImage);
cv::waitKey(0);
Try using cvFindContours() function. You can use it to find connected components. With the right parameters this function returns a list with the contours of each connected components.
Find the contours which represent a hole. Then use cvDrawContours() to fill up the selected contour by the foreground color thereby closing the holes.
I think if the objects are touched or crowded, there will be some problems using the contours and the math morophology opening.
Instead, the following simple solution is found and tested. It is working very well, and not only for this images, but also for any other images.
here is the steps (optimized) as seen in http://blogs.mathworks.com/steve/2008/08/05/filling-small-holes/
let I: the input image
1. filled_I = floodfill(I). // fill every hole in the image.
2. inverted_I = invert(I)`.
3. holes_I = filled_I AND inverted_I. // finds all holes
4. cc_list = connectedcomponent(holes_I) // list of all connected component in holes_I.
5. holes_I = remove(cc_list,holes_I, smallholes_threshold_size) // remove all holes from holes_I having size > smallholes_threshold_size.
6. out_I = I OR holes_I. // fill only the small holes.
In short, the algorithm is just to find all holes, remove the big ones then write the small ones only on the original image.
I've been looking around the internet to find a proper imfill function (as the one in Matlab) but working in C with OpenCV. After some reaserches, I finally came up with a solution :
IplImage* imfill(IplImage* src)
{
CvScalar white = CV_RGB( 255, 255, 255 );
IplImage* dst = cvCreateImage( cvGetSize(src), 8, 3);
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* contour = 0;
cvFindContours(src, storage, &contour, sizeof(CvContour), CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
cvZero( dst );
for( ; contour != 0; contour = contour->h_next )
{
cvDrawContours( dst, contour, white, white, 0, CV_FILLED);
}
IplImage* bin_imgFilled = cvCreateImage(cvGetSize(src), 8, 1);
cvInRangeS(dst, white, white, bin_imgFilled);
return bin_imgFilled;
}
For this: Original Binary Image
Result is: Final Binary Image
The trick is in the parameters setting of the cvDrawContours function:
cvDrawContours( dst, contour, white, white, 0, CV_FILLED);
dst = destination image
contour = pointer to the first contour
white = color used to fill the contour
0 = Maximal level for drawn contours. If 0, only contour is drawn
CV_FILLED = Thickness of lines the contours are drawn with. If it is negative (For example, =CV_FILLED), the contour interiors are drawn.
More info in the openCV documentation.
There is probably a way to get "dst" directly as a binary image but I couldn't find how to use the cvDrawContours function with binary values.