Related
I am working on detecting the center and radius of a circular aperture that is illuminated by a laser beam. The algorithm will be fed images from a system that I have no physical control over (i.e. dimming the source or adjusting the laser position.) I need to do this with C++, and have chosen to use openCV.
In some regions the edge of the aperture is well defined, but in others it is very noisy. I currently am trying to isolate the "good" points to do a RANSAC fit, but I have taken other steps along the way. Below are two original images for reference:
I first began by trying to do a Hough fit. I performed a median blur to remove the salt and pepper noise, then a Gaussian blur, and then fed the image to the HoughCircle function in openCV, with sliders controlling the Hough parameters 1 and 2 defined here. The results were disastrous:
I then decided to try to process the image some more before sending it to the HoughCircle. I started with the original image, median blurred, Gaussian blurred, thresholded, dilated, did a Canny edge detection, and then fed the Canny image to the function.
I was eventually able to get a reasonable estimate of my circle, but it was about the 15th circle to show up when manually decreasing the Hough parameters. I manually drew the purple outline, with the green circles representing Hough outputs that were near my manual estimate. The below images are:
Canny output without dilation
Canny output with dilation
Hough output of the dilated Canny image drawn on the original image.
As you can see, the number of invalid circles vastly outnumbers the correct circle, and I'm not quite sure how to isolate the good circles given that the Hough transform returns so many other invalid circles with parameters that are more strict.
I currently have some code I implemented that works OK for all of the test images I was given, but the code is a convoluted mess with many tunable parameters that seems very fragile. The driving logic behind what I did was from noticing that regions of the aperture edges that were well-illuminated by the laser were relatively constant across several threshold levels (image shown below).
I did edge detection at two threshold levels and stored points that overlapped in both images. Currently there is also some inaccuracy with the result because the aperture edge does still shift slightly with the different threshold levels. I can post the very long code for this if necessary, but the pseudo-code behind it is:
1. Perform a median blur, followed by a Gaussian blur. Kernels are 9x9.
2. Threshold the image until 35% of the image is white. (~intensities > 30)
3. Take the Canny edges of this thresholded image and store (Canny1)
4. Take the original image, perform the same median and Gaussian blurs, but threshold with a 50% larger value, giving a smaller spot (~intensities > 45)
5. Perform the "Closing" morphology operation to further erode the spot and remove any smaller contours.
6. Perform another Canny to get the edges, and store this image (Canny2)
7. Blur both the Canny images with a 7x7 Gaussian blur.
8. Take the regions where the two Canny images overlap and say that these points are likely to be good points.
9. Do a RANSAC circle fit with these points.
I've noticed that there are regions of the edge detected circle that are pretty distinguishable by the human eye as being part of the best circle. Is there a way to isolate these regions for a RANSAC fit?
Code for Hough:
int houghParam1 = 100;
int houghParam2 = 100;
int dp = 10; //divided by 10 later
int x=616;
int y=444;
int radius = 398;
int iterations = 0;
int main()
{
namedWindow("Circled Orig");
namedWindow("Processed", 1);
namedWindow("Circles");
namedWindow("Parameters");
namedWindow("Canny");
createTrackbar("Param1", "Parameters", &houghParam1, 200);
createTrackbar("Param2", "Parameters", &houghParam2, 200);
createTrackbar("dp", "Parameters", &dp, 20);
createTrackbar("x", "Parameters", &x, 1200);
createTrackbar("y", "Parameters", &y, 1200);
createTrackbar("radius", "Parameters", &radius, 900);
createTrackbar("dilate #", "Parameters", &iterations, 20);
std::string directory = "Secret";
std::string suffix = ".pgm";
Mat processedImage;
Mat origImg;
for (int fileCounter = 2; fileCounter < 3; fileCounter++) //1, 12
{
std::string numString = std::to_string(static_cast<long long>(fileCounter));
std::string imageFile = directory + numString + suffix;
testImage = imread(imageFile);
Mat bwImage;
cvtColor(testImage, bwImage, CV_BGR2GRAY);
GaussianBlur(bwImage, processedImage, Size(9, 9), 9);
threshold(processedImage, processedImage, 25, 255, THRESH_BINARY); //THRESH_OTSU
int numberContours = -1;
int iterations = 1;
imshow("Processed", processedImage);
}
vector<Vec3f> circles;
Mat element = getStructuringElement(MORPH_ELLIPSE, Size(5, 5));
float dp2 = dp;
while (true)
{
float dp2 = dp;
Mat circleImage = processedImage.clone();
origImg = testImage.clone();
if (iterations > 0) dilate(circleImage, circleImage, element, Point(-1, -1), iterations);
Mat cannyImage;
Canny(circleImage, cannyImage, 100, 20);
imshow("Canny", cannyImage);
HoughCircles(circleImage, circles, HOUGH_GRADIENT, dp2/10, 5, houghParam1, houghParam2, 300, 5000);
cvtColor(circleImage, circleImage, CV_GRAY2BGR);
for (size_t i = 0; i < circles.size(); i++)
{
Scalar color = Scalar(0, 0, 255);
Point center2(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius2 = cvRound(circles[i][2]);
if (abs(center2.x - x) < 10 && abs((center2.y - y) < 10) && abs(radius - radius2) < 20) color = Scalar(0, 255, 0);
circle(circleImage, center2, 3, color, -1, 8, 0);
circle(circleImage, center2, radius2, color, 3, 8, 0);
circle(origImg, center2, 3, color, -1, 8, 0);
circle(origImg, center2, radius2,color, 3, 8, 0);
}
//Manual circles
circle(circleImage, Point(x, y), 3, Scalar(128, 0, 128), -1, 8, 0);
circle(circleImage, Point(x, y), radius, Scalar(128, 0, 128), 3, 8, 0);
circle(origImg, Point(x, y), 3, Scalar(128, 0, 128), -1, 8, 0);
circle(origImg, Point(x, y), radius, Scalar(128, 0, 128), 3, 8, 0);
imshow("Circles", circleImage);
imshow("Circled Orig", origImg);
int x = waitKey(50);
}
Mat drawnImage;
cvtColor(processedImage, drawnImage, CV_GRAY2BGR);
return 1;
}
Thanks #jalconvolvon - this is an interesting problem. Here's my result:
What I find important on and on is using dynamic parameter adjustment when prototyping, thus I include the function I used to tune Canny detection. The code also uses this answer for the Ransac part.
import cv2
import numpy as np
import auxcv as aux
from skimage import measure, draw
def empty_function(*arg):
pass
# tune canny edge detection. accept with pressing "C"
def CannyTrackbar(img, win_name):
trackbar_name = win_name + "Trackbar"
cv2.namedWindow(win_name)
cv2.resizeWindow(win_name, 500,100)
cv2.createTrackbar("canny_th1", win_name, 0, 255, empty_function)
cv2.createTrackbar("canny_th2", win_name, 0, 255, empty_function)
cv2.createTrackbar("blur_size", win_name, 0, 255, empty_function)
cv2.createTrackbar("blur_amp", win_name, 0, 255, empty_function)
while True:
trackbar_pos1 = cv2.getTrackbarPos("canny_th1", win_name)
trackbar_pos2 = cv2.getTrackbarPos("canny_th2", win_name)
trackbar_pos3 = cv2.getTrackbarPos("blur_size", win_name)
trackbar_pos4 = cv2.getTrackbarPos("blur_amp", win_name)
img_blurred = cv2.GaussianBlur(img.copy(), (trackbar_pos3 * 2 + 1, trackbar_pos3 * 2 + 1), trackbar_pos4)
canny = cv2.Canny(img_blurred, trackbar_pos1, trackbar_pos2)
cv2.imshow(win_name, canny)
key = cv2.waitKey(1) & 0xFF
if key == ord("c"):
break
cv2.destroyAllWindows()
return canny
img = cv2.imread("sphere.jpg")
#resize for convenience
img = cv2.resize(img, None, fx = 0.2, fy = 0.2)
#closing
kernel = np.ones((11,11), np.uint8)
img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)
#sharpening
kernel = np.array([[-1,-1,-1], [-1,9,-1], [-1,-1,-1]])
img = cv2.filter2D(img, -1, kernel)
#test if you use different scale img than 0.2 of the original that I used
#remember that the actual kernel size for GaussianBlur is trackbar_pos3*2+1
#you want to get as full circle as possible here
#canny = CannyTrackbar(img, "canny_trakbar")
#additional blurring to reduce the offset toward brighter region
img_blurred = cv2.GaussianBlur(img.copy(), (8*2+1,8*2+1), 1)
#detect edge. important: make sure this works well with CannyTrackbar()
canny = cv2.Canny(img_blurred, 160, 78)
coords = np.column_stack(np.nonzero(canny))
model, inliers = measure.ransac(coords, measure.CircleModel,
min_samples=3, residual_threshold=1,
max_trials=1000)
rr, cc = draw.circle_perimeter(int(model.params[0]),
int(model.params[1]),
int(model.params[2]),
shape=img.shape)
img[rr, cc] = 1
import matplotlib.pyplot as plt
plt.imshow(img, cmap='gray')
plt.scatter(model.params[1], model.params[0], s=50, c='red')
plt.axis('off')
plt.savefig('sphere_center.png', bbox_inches='tight')
plt.show()
Now I'd probably try to calculate where pixels are statisticaly brigher and where they are dimmer to adjust the laser position (if I understand correctly what you're trying to do)
If the Ransac is still not enough. I'd try tuning Canny to only detect a perfect arc on top of the circle (where it's well outlined) and than try using the following dependencies (I suspect that this should be possible):
We're currently trying to detect the object regions in medical instruments images using the methods available in OpenCV, C++ version. An example image is shown below:
Here are the steps we're following:
Converting the image to gray scale
Applying median filter
Find edges using sobel filter
Convert the result to binary image using a threshold of 25
Skeletonize the image to make sure we have neat edges
Finding X largest connected components
This approach works perfectly for the image 1 and here is the result:
The yellow borders are the connected components detected.
The rectangles are just to highlight the presence of a connected component.
To get understandable results, we just removed the connected components that are completely inside any another one, so the end result is something like this:
So far, everything was fine but another sample of image complicated our work shown below.
Having a small light green towel under the objects results this image:
After filtering the regions as we did earlier, we got this:
Obviously, it is not what we need..we're excepting something like this:
I'm thinking about clustering the closest connected components found(somehow!!) so we can minimize the impact of the presence of the towel, but don't know yet if it's something doable or someone has tried something like this before? Also, does anyone have any better idea to overcome this kind of problems?
Thanks in advance.
Here's what I tried.
In the images, the background is mostly greenish and the area of the background is considerably larger than that of the foreground. So, if you take a color histogram of the image, the greenish bins will have higher values. Threshold this histogram so that bins having smaller values are set to zero. This way we'll most probably retain the greenish (higher value) bins and discard other colors. Then backproject this histogram. The backprojection will highlight these greenish regions in the image.
Backprojection:
Then threshold this backprojection. This gives us the background.
Background (after some morphological filtering):
Invert the background to get foreground.
Foreground (after some morphological filtering):
Then find the contours of the foreground.
I think this gives a reasonable segmentation, and using this as mask you may be able to use a segmentation like GrabCut to refine the boundaries (I haven't tried this yet).
EDIT:
I tried the GrabCut approach and it indeed refines the boundaries. I've added the code for GrabCut segmentation.
Contours:
GrabCut segmentation using the foreground as mask:
I'm using the OpenCV C API for the histogram processing part.
// load the color image
IplImage* im = cvLoadImage("bFly6.jpg");
// get the color histogram
IplImage* im32f = cvCreateImage(cvGetSize(im), IPL_DEPTH_32F, 3);
cvConvertScale(im, im32f);
int channels[] = {0, 1, 2};
int histSize[] = {32, 32, 32};
float rgbRange[] = {0, 256};
float* ranges[] = {rgbRange, rgbRange, rgbRange};
CvHistogram* hist = cvCreateHist(3, histSize, CV_HIST_ARRAY, ranges);
IplImage* b = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* g = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* r = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* backproject32f = cvCreateImage(cvGetSize(im), IPL_DEPTH_32F, 1);
IplImage* backproject8u = cvCreateImage(cvGetSize(im), IPL_DEPTH_8U, 1);
IplImage* bw = cvCreateImage(cvGetSize(im), IPL_DEPTH_8U, 1);
IplConvKernel* kernel = cvCreateStructuringElementEx(3, 3, 1, 1, MORPH_ELLIPSE);
cvSplit(im32f, b, g, r, NULL);
IplImage* planes[] = {b, g, r};
cvCalcHist(planes, hist);
// find min and max values of histogram bins
float minval, maxval;
cvGetMinMaxHistValue(hist, &minval, &maxval);
// threshold the histogram. this sets the bin values that are below the threshold to zero
cvThreshHist(hist, maxval/32);
// backproject the thresholded histogram. backprojection should contain higher values for the
// background and lower values for the foreground
cvCalcBackProject(planes, backproject32f, hist);
// convert to 8u type
double min, max;
cvMinMaxLoc(backproject32f, &min, &max);
cvConvertScale(backproject32f, backproject8u, 255.0 / max);
// threshold backprojected image. this gives us the background
cvThreshold(backproject8u, bw, 10, 255, CV_THRESH_BINARY);
// some morphology on background
cvDilate(bw, bw, kernel, 1);
cvMorphologyEx(bw, bw, NULL, kernel, MORPH_CLOSE, 2);
// get the foreground
cvSubRS(bw, cvScalar(255, 255, 255), bw);
cvMorphologyEx(bw, bw, NULL, kernel, MORPH_OPEN, 2);
cvErode(bw, bw, kernel, 1);
// find contours of the foreground
//CvMemStorage* storage = cvCreateMemStorage(0);
//CvSeq* contours = 0;
//cvFindContours(bw, storage, &contours);
//cvDrawContours(im, contours, CV_RGB(255, 0, 0), CV_RGB(0, 0, 255), 1, 2);
// grabcut
Mat color(im);
Mat fg(bw);
Mat mask(bw->height, bw->width, CV_8U);
mask.setTo(GC_PR_BGD);
mask.setTo(GC_PR_FGD, fg);
Mat bgdModel, fgdModel;
grabCut(color, mask, Rect(), bgdModel, fgdModel, GC_INIT_WITH_MASK);
Mat gcfg = mask == GC_PR_FGD;
vector<vector<cv::Point>> contours;
vector<Vec4i> hierarchy;
findContours(gcfg, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
for(int idx = 0; idx < contours.size(); idx++)
{
drawContours(color, contours, idx, Scalar(0, 0, 255), 2);
}
// cleanup ...
UPDATE: We can do the above using the C++ interface as shown below.
const int channels[] = {0, 1, 2};
const int histSize[] = {32, 32, 32};
const float rgbRange[] = {0, 256};
const float* ranges[] = {rgbRange, rgbRange, rgbRange};
Mat hist;
Mat im32fc3, backpr32f, backpr8u, backprBw, kernel;
Mat im = imread("bFly6.jpg");
im.convertTo(im32fc3, CV_32FC3);
calcHist(&im32fc3, 1, channels, Mat(), hist, 3, histSize, ranges, true, false);
calcBackProject(&im32fc3, 1, channels, hist, backpr32f, ranges);
double minval, maxval;
minMaxIdx(backpr32f, &minval, &maxval);
threshold(backpr32f, backpr32f, maxval/32, 255, THRESH_TOZERO);
backpr32f.convertTo(backpr8u, CV_8U, 255.0/maxval);
threshold(backpr8u, backprBw, 10, 255, THRESH_BINARY);
kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
dilate(backprBw, backprBw, kernel);
morphologyEx(backprBw, backprBw, MORPH_CLOSE, kernel, Point(-1, -1), 2);
backprBw = 255 - backprBw;
morphologyEx(backprBw, backprBw, MORPH_OPEN, kernel, Point(-1, -1), 2);
erode(backprBw, backprBw, kernel);
Mat mask(backpr8u.rows, backpr8u.cols, CV_8U);
mask.setTo(GC_PR_BGD);
mask.setTo(GC_PR_FGD, backprBw);
Mat bgdModel, fgdModel;
grabCut(im, mask, Rect(), bgdModel, fgdModel, GC_INIT_WITH_MASK);
Mat fg = mask == GC_PR_FGD;
I would consider a few options. My assumption is that the camera does not move. I haven't used the images or written any code, so this is mostly from experience.
Rather than just looking for edges, try separating the background using a segmentation algorithm. Mixture of Gaussian can help with this. Given a set of images over the same region (i.e. video), you can cancel out regions which are persistent. Then, new items such as instruments will pop out. Connected components can then be used on the blobs.
I would look at segmentation algorithms to see if you can optimize the conditions to make this work for you. One major item is to make sure your camera is stable or you stabilize the images yourself pre-processing.
I would consider using interest points to identify regions in the image with a lot of new material. Given that the background is relatively plain, small objects such as needles will create a bunch of interest points. The towel should be much more sparse. Perhaps overlaying the detected interest points over the connected component footprint will give you a "density" metric which you can then threshold. If the connected component has a large ratio of interest points for the area of the item, then it is an interesting object.
On this note, you can even clean up the connected component footprint by using a Convex Hull to prune the objects you have detected. This may help situations such as a medical instrument casting a shadow on the towel which stretches the component region. This is a guess, but interest points can definitely give you more information than just edges.
Finally, given that you have a stable background with clear objects in view, I would take a look at Bag-of-Features to see if you can just detect each individual object in the image. This may be useful since there seems to be a consistent pattern to the objects in these images. You can build a big database of images such as needles, gauze, scissors, etc. Then BoF, which is in OpenCV will find those candidates for you. You can also mix it in with other operations you are doing to compare results.
Bag of Features using OpenCV
http://www.codeproject.com/Articles/619039/Bag-of-Features-Descriptor-on-SIFT-Features-with-O
-
I would also suggest an idea to your initial version. You can also skip the contours, whose regions have width and height greater than the half the image width and height.
//take the rect of the contours
Rect rect = Imgproc.boundingRect(contours.get(i));
if (rect.width < inputImageWidth / 2 && rect.height < inputImageHeight / 2)
//then continue to draw or use for next purposes.
I have a color image that I want to a threshold in OpenCV. What I would like is that if any of the RGB channels in under a certain value, to set the value in all the channels to zero (i.e. black).
So, I use the opencv threshold function as:
cv::Mat frame, thresholded
// read frame somewhere, it is a BGR image.
cv::threshold(frame, thresholded, 5, 255, cv::THRESH_BINARY);
So, what I thought this would do is that if any of the channels is less than 5, I thought it would set them to zero. However, it does not seem to work that way. For example, I see only the green channel come through for some of these regions, indicating not all channels are set to 0.
Is there a way to achieve this using OpenCV in a fast way?
It's possible to threshold a colored image using the function cv::inRange.
void inRange(InputArray src, InputArray lowerb, InputArray upperb, OutputArray dst)
For example, you can allow only values between (0, 125, 0) and (255, 200, 255), or any values for individual channels:
cv::Mat image = cv::imread("bird.jpg");
if (image.empty())
{
std::cout << "> This image is empty" << std::endl;
return 1;
}
cv::Mat output;
cv::inRange(image, cv::Scalar(0, 125, 0), cv::Scalar(255, 200, 255), output);
cv::imshow("output", output);
In short, you have to slipt your image in three images containing the three channels, threeshold them independantly and then merge them again.
Mat frame,thresholded;
vector<Mat> splited_frame;
//Read your frame
split(frame, splited_frame);
for (size_t i = 0; i < splited_frame.size(); i++)
threshold(splited_frame[i], splited_frame[i], 5, 255, cv::THRESH_BINARY);
merge(splited_frame,thresholded);
This code should do it.
Sorry, I read to fast.
Then, you should modify the code slightly after the for
thresholded = splited_frame[0].clone();
for(size_t i = 1; i < splited_frame.size(); i++) thresholded &= splited_frame[i];
frame &= thresholded;
You create a mask from the three thresholded images, then apply this mask to your input image.
I want to compute the red circles radius (fig 2). I have troubles finding these circles using HoughCircles from OpenCV. As you can see in fig. 2 I can only find the little circles in center which are shown in black using HoughCircles.
original fig 2.
Since I know the center of the red circles (which are the same as the red ones), is there a way to compute simply the radius of the red circles ?
Is it also possible to have a generic way of computing radius of circles on a more complex image such as this one :
Edit : Here the interesting part of my code after obtaining fig 2 :
threshold(maskedImage, maskedImage, thresh, 255, THRESH_BINARY_INV | THRESH_OTSU);
std::vector<Vec3f> circles;
// Canny(maskedImage, maskedImage, thresh, thresh * 2, 3);
HoughCircles(maskedImage, circles, CV_HOUGH_GRADIENT, 1, src_gray.rows / 4, cannyThreshold, accumulatorThreshold, 0, 0);
Mat display = src_display.clone();
for (size_t i = 0; i < circles.size(); i++)
{
Point center(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius = cvRound(circles[i][2]);
// circle center
circle(display, center, 3, Scalar(0, 255, 0), -1, 8, 0);
// circle outline
circle(display, center, radius, Scalar(0, 0, 255), 3, 8, 0);
}
I have tried to use play with cannyThreshold and accumulator without results. Real images are 5x biggers. Here a link for example 1 after threshold.
Thanks
You already know the smaller circles in the image(which you have drawn in black).
Prepare a mask image using these circles so the areas having smaller circles will have non-zero pixels. We'll call it mask:
In the original image, fill these circle areas in a dark color(say black). This will result in an image like your fig 2. We'll call it filled
Threshold the filled image to obtain the dark areas. We'll call it binary. You can use Otsu thresholding for this. Result will look something like this:
Take the distance transform of this binary image. Use an accurate distance estimation method for this. We'll call this dist. It'll look something like this. The colored one is just a heat map for more clarity:
Use the mask to obtain the peak regions from dist. The max value of each such region should give you the radius of the larger circle. You can also do some processing on these regions to arrive at a more reasonable value for radius rather than just picking up the max.
For selecting the regions, you can either find the contours of the mask and then extract that region from dist image, or, since you already know the smaller circles from applying hough-circle transform, prepare a mask from each of those circles and extract that region from dist image. I'm not sure if you can calculate max or other stats by giving a mask. Max will definitely work because the rest of the pixels are 0. You might be able calculate the stats of the region if you extract those pixels to another array.
Figures below show such mask and the extracted region from dist. For this I get a max around 29 which is consistent with the radius of that circle. Note that the images are not to scale.
mask for a circle, extracted region from dist
Here's the code (I'm not using hough-circles transform):
Mat im = imread(INPUT_FOLDER_PATH + string("ex1.jpg"));
Mat gray;
cvtColor(im, gray, CV_BGR2GRAY);
Mat bw;
threshold(gray, bw, 0, 255, CV_THRESH_BINARY|CV_THRESH_OTSU);
// filtering smaller circles: not using hough-circles transform here.
// you can replace this part with you hough-circles code.
vector<int> circles;
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(bw, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
for(int idx = 0; idx >= 0; idx = hierarchy[idx][0])
{
Rect rect = boundingRect(contours[idx]);
if (abs(1.0 - ((double)rect.width/rect.height) < .1))
{
Mat mask = Mat::zeros(im.rows, im.cols, CV_8U);
drawContours(mask, contours, idx, Scalar(255, 255, 255), -1);
double area = sum(mask).val[0]/255;
double rad = (rect.width + rect.height)/4.0;
double circArea = CV_PI*rad*rad;
double dif = abs(1.0 - area/circArea);
if (dif < .5 && rad < 50 && rad > 30) // restrict the radius
{
circles.push_back(idx); // store smaller circle contours
drawContours(gray, contours, idx, Scalar(0, 0, 0), -1); // fill circles
}
}
}
threshold(gray, bw, 0, 255, CV_THRESH_BINARY_INV|CV_THRESH_OTSU);
Mat dist, distColor, color;
distanceTransform(bw, dist, CV_DIST_L2, 5);
double max;
Point maxLoc;
minMaxLoc(dist, NULL, &max);
dist.convertTo(distColor, CV_8U, 255.0/max);
applyColorMap(distColor, color, COLORMAP_JET);
imshow("", color);
waitKey();
// extract dist region corresponding to each smaller circle and find max
for(int idx = 0; idx < (int)circles.size(); idx++)
{
Mat masked;
Mat mask = Mat::zeros(im.rows, im.cols, CV_8U);
drawContours(mask, contours, circles[idx], Scalar(255, 255, 255), -1);
dist.copyTo(masked, mask);
minMaxLoc(masked, NULL, &max, NULL, &maxLoc);
circle(im, maxLoc, 4, Scalar(0, 255, 0), -1);
circle(im, maxLoc, (int)max, Scalar(0, 0, 255), 2);
cout << "rad: " << max << endl;
}
imshow("", im);
waitKey();
Results(scaled):
Hope this helps.
I know that there are a hundred topics about my question in all over the web, but i would like to ask specific for my problem because I tried almost all solutions without any success.
I am trying to count circles in an image (yes i have already tried hough circles but due to light reflections, i think, on my object is not very robust).
Then I tried to create a classifier (no success i think there is no enough features so the detection is not good)
I have also tried HSV conversation and tried to find my object with color (again I had some problems because of the light and the variations of colors)
As you can see on image, there are 8 caps and i would like to be able to count them.
Using all of this methods, i was able to detect the objects on an image (because I was optimizing all the parameters of functions for the specific image) but as soon as I load a new, similar, image the results was disappointing.
Please follow this link to see the Image
Bellow you can find parts of everything i have tried:
1. Hough circles
img = cv2.imread('frame71.jpg',1)
img = cv2.medianBlur(img,5)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
if img == None:
print "There is no image file. Quiting..."
quit()
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,3,50,
param1=55,param2=125,minRadius=25,maxRadius=45)
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
# draw the outer circle
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),2)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
print len(circles[0,:])
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
2. HSV Transform, color detection
def image_process(frame, h_low, s_low, v_low, h_up, s_up, v_up, ksize):
temp = ksize
if(temp%2==1):
ksize = temp
else:
ksize = temp+1
#if(True):
# return frame
#thresh = frame
#try:
#TODO: optimize as much as possiblle this part of code
try:
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
lower = np.array([h_low, s_low, v_low],np.uint8)
upper = np.array([h_up,s_up,h_up],np.uint8)
mask = cv2.inRange(hsv, lower, upper)
res = cv2.bitwise_and(hsv,hsv, mask= mask)
thresh = cv2.cvtColor(res, cv2.COLOR_BGR2GRAY)
#thresh = cv2.threshold(res, 50, 255, cv2.THRESH_BINARY)[1]
thresh = cv2.threshold(thresh, 50, 255, cv2.THRESH_BINARY)[1]
thresh = cv2.medianBlur(thresh,ksize)
except Exception as inst:
print type(inst)
#cv2.imshow('thresh', thresh)
return thresh
3. Cascade classifier
img = cv2.imread('frame405.jpg', 1)
cap_cascade = cv2.CascadeClassifier('haar_30_17_16_stage.xml')
caps = cap_cascade.detectMultiScale(img, 1.3, 5)
#print caps
for (x,y,w,h) in caps:
cv2.rectangle(img, (x,y), (x+w,y+h), (255,0,0),2)
#cv2.rectangle(img, (10,10),(100,100),(0,255,255),4)
cv2.imshow('image', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
quit()
About training the classifier I really used a lot of variations of images, samples, negatives and positives, number of stages, w and h but the results was not very accurate.
Finally I would like to know from your experience which is the best method I should follow and I will stick on that in order to optimize my detection. Keep in mind that all images are similiar but NOT identical. There are some differences due to light, movement etc
Than you in advance,
I did some experiment with the sample image. I'm posting my results, and if you find it useful, you can improve it further and optimize. Here are the steps:
downsample the image
perform morphological opening
find Hough circles
cluster the circles by radii (bottle circles should get the same label)
filter the circles by a radius threshold
you can also cluster circles by their center x and y coordinates (I haven't done this)
prepare a mask from the filtered circles and extract the possible bottles region
cluster this region by color
Code is in C++. I'm attaching my results.
Mat im = imread(INPUT_FOLDER_PATH + string("frame71.jpg"));
Mat small;
int kernelSize = 9; // try with different kernel sizes. 5 onwards gives good results
pyrDown(im, small); // downsample the image
Mat morph;
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(kernelSize, kernelSize));
morphologyEx(small, morph, MORPH_OPEN, kernel); // open
Mat gray;
cvtColor(morph, gray, CV_BGR2GRAY);
vector<Vec3f> circles;
HoughCircles(gray, circles, CV_HOUGH_GRADIENT, 2, gray.rows/8.0); // find circles
// -------------------------------------------------------
// cluster the circles by radii. similarly you can cluster them by center x and y for further filtering
Mat circ = Mat(circles);
Mat data[3];
split(circ, data);
Mat labels, centers;
kmeans(data[2], 2, labels, TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 10, 1.0), 2, KMEANS_PP_CENTERS, centers);
// -------------------------------------------------------
Mat rgb;
small.copyTo(rgb);
//cvtColor(gray, rgb, CV_GRAY2BGR);
Mat mask = Mat::zeros(Size(gray.cols, gray.rows), CV_8U);
for(size_t i = 0; i < circles.size(); i++)
{
Point center(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius = cvRound(circles[i][2]);
float r = centers.at<float>(labels.at<int>(i));
if (r > 30.0f && r < 45.0f) // filter circles by radius (values are based on the sample image)
{
// just for display
circle(rgb, center, 3, Scalar(0,255,0), -1, 8, 0);
circle(rgb, center, radius, Scalar(0,0,255), 3, 8, 0);
// prepare a mask
circle(mask, center, radius, Scalar(255,255,255), -1, 8, 0);
}
}
// use each filtered circle as a mask and extract the region from original downsampled image
Mat rgb2;
small.copyTo(rgb2, mask);
// cluster the masked region by color
Mat rgb32fc3, lbl;
rgb2.convertTo(rgb32fc3, CV_32FC3);
int imsize[] = {rgb32fc3.rows, rgb32fc3.cols};
Mat color = rgb32fc3.reshape(1, rgb32fc3.rows*rgb32fc3.cols);
kmeans(color, 4, lbl, TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 10, 1.0), 2, KMEANS_PP_CENTERS);
Mat lbl2d = lbl.reshape(1, 2, imsize);
Mat lbldisp;
lbl2d.convertTo(lbldisp, CV_8U, 50);
Mat lblColor;
applyColorMap(lbldisp, lblColor, COLORMAP_JET);
Results:
Filtered circles:
Masked:
Segmented:
Hello finally i think I found a way to count caps on bottles.
Read image
Teach (find correct values for HSV up/low limits)
Select desire color (using HSV and mask)
Find contours on the masked image
Find the minCircles for contours
Reject all circles beyond thresholds
I have also order a polarized filter which I think it will reduce glares a lot. I am open to suggestions for further improvement (robustness and speed). Both robustness and speed is crucial for my application.
Thank you.