I'm trying to count the number of erythrocytes on a microscope image. These are the smaller cells. (I've tried first using CNN and sliding window, but it was too slow, so I'm looking for a simplier segmentation)
My approach is:
threshold
find and draw all contours filled so that the cells won't have holes,
make distance transform
iterating over all maxima
masking out a current maximum with a circle having the radius of the maximum and storing the maximum position
My problem is, some cells have a "hole" in the middle - bright area similar by the value to background. If I threshold the image, some of the cell-masks become not a circle but a half circle, with the distance-transform values far below expected value.
I've marked the cells having the "holes" on the mask image.
Hov could I close the hole or the circle? Is there a threshold method or trick?
Below is the part of code responsible for cell extraction:
cv::adaptiveThreshold(_imgIn ,th, 255, ADAPTIVE_THRESH_GAUSSIAN_C, (bgblack ? CV_THRESH_BINARY: CV_THRESH_BINARY_INV), 35, 5 );//| CV_THRESH_OTSU);
Mat kernel1 = Mat::ones(3, 3, CV_8UC1);
for (int i=0; i< 5;i++)
{
dilate(th, th, kernel1);
erode(th, th, kernel1);
}
vector<vector<Point> > contours;
findContours(th, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
mask = 0;
for( unsigned int i = 0; i < contours.size(); i++ )
{
drawContours(mask, contours, i, Scalar(255), CV_FILLED);
}
cv::distanceTransform(mask, dist, CV_DIST_L2, 3);
}
double min, max;
cv::Point pmax;
Mat tmp1 = dist.clone();
while (true)
{
cv::minMaxLoc(tmp1, 0, &max, 0, &pmax);
if ( max < 5 )
break;
cv::circle(_imgIn, pmax, 3 , cv::Scalar(0), CV_FILLED );
cv::circle(tmp1, pmax, max , cv::Scalar(0), CV_FILLED );
}
Closing holes
Closing is an important operator from the field of mathematical morphology. Like its dual operator opening, it can be derived from the fundamental operations of erosion and dilation. Like those operators it is normally applied to binary images, although there are graylevel versions. Closing is similar in some ways to dilation in that it tends to enlarge the boundaries of foreground (bright) regions in an image (and shrink background color holes in such regions), but it is less destructive of the original boundary shape. As with other morphological operators, the exact operation is determined by a structuring element. The effect of the operator is to preserve background regions that have a similar shape to this structuring element, or that can completely contain the structuring element, while eliminating all other regions of background pixels.
In Open CV this looks as follows
import cv2 as cv
import numpy as np
img = cv.imread('j.png',0)
kernel = np.ones((5,5),np.uint8)
erosion = cv.erode(img,kernel,iterations = 1)
closing = cv.morphologyEx(img, cv.MORPH_CLOSE, kernel)
Full documentation here.
I am working on detecting the center and radius of a circular aperture that is illuminated by a laser beam. The algorithm will be fed images from a system that I have no physical control over (i.e. dimming the source or adjusting the laser position.) I need to do this with C++, and have chosen to use openCV.
In some regions the edge of the aperture is well defined, but in others it is very noisy. I currently am trying to isolate the "good" points to do a RANSAC fit, but I have taken other steps along the way. Below are two original images for reference:
I first began by trying to do a Hough fit. I performed a median blur to remove the salt and pepper noise, then a Gaussian blur, and then fed the image to the HoughCircle function in openCV, with sliders controlling the Hough parameters 1 and 2 defined here. The results were disastrous:
I then decided to try to process the image some more before sending it to the HoughCircle. I started with the original image, median blurred, Gaussian blurred, thresholded, dilated, did a Canny edge detection, and then fed the Canny image to the function.
I was eventually able to get a reasonable estimate of my circle, but it was about the 15th circle to show up when manually decreasing the Hough parameters. I manually drew the purple outline, with the green circles representing Hough outputs that were near my manual estimate. The below images are:
Canny output without dilation
Canny output with dilation
Hough output of the dilated Canny image drawn on the original image.
As you can see, the number of invalid circles vastly outnumbers the correct circle, and I'm not quite sure how to isolate the good circles given that the Hough transform returns so many other invalid circles with parameters that are more strict.
I currently have some code I implemented that works OK for all of the test images I was given, but the code is a convoluted mess with many tunable parameters that seems very fragile. The driving logic behind what I did was from noticing that regions of the aperture edges that were well-illuminated by the laser were relatively constant across several threshold levels (image shown below).
I did edge detection at two threshold levels and stored points that overlapped in both images. Currently there is also some inaccuracy with the result because the aperture edge does still shift slightly with the different threshold levels. I can post the very long code for this if necessary, but the pseudo-code behind it is:
1. Perform a median blur, followed by a Gaussian blur. Kernels are 9x9.
2. Threshold the image until 35% of the image is white. (~intensities > 30)
3. Take the Canny edges of this thresholded image and store (Canny1)
4. Take the original image, perform the same median and Gaussian blurs, but threshold with a 50% larger value, giving a smaller spot (~intensities > 45)
5. Perform the "Closing" morphology operation to further erode the spot and remove any smaller contours.
6. Perform another Canny to get the edges, and store this image (Canny2)
7. Blur both the Canny images with a 7x7 Gaussian blur.
8. Take the regions where the two Canny images overlap and say that these points are likely to be good points.
9. Do a RANSAC circle fit with these points.
I've noticed that there are regions of the edge detected circle that are pretty distinguishable by the human eye as being part of the best circle. Is there a way to isolate these regions for a RANSAC fit?
Code for Hough:
int houghParam1 = 100;
int houghParam2 = 100;
int dp = 10; //divided by 10 later
int x=616;
int y=444;
int radius = 398;
int iterations = 0;
int main()
{
namedWindow("Circled Orig");
namedWindow("Processed", 1);
namedWindow("Circles");
namedWindow("Parameters");
namedWindow("Canny");
createTrackbar("Param1", "Parameters", &houghParam1, 200);
createTrackbar("Param2", "Parameters", &houghParam2, 200);
createTrackbar("dp", "Parameters", &dp, 20);
createTrackbar("x", "Parameters", &x, 1200);
createTrackbar("y", "Parameters", &y, 1200);
createTrackbar("radius", "Parameters", &radius, 900);
createTrackbar("dilate #", "Parameters", &iterations, 20);
std::string directory = "Secret";
std::string suffix = ".pgm";
Mat processedImage;
Mat origImg;
for (int fileCounter = 2; fileCounter < 3; fileCounter++) //1, 12
{
std::string numString = std::to_string(static_cast<long long>(fileCounter));
std::string imageFile = directory + numString + suffix;
testImage = imread(imageFile);
Mat bwImage;
cvtColor(testImage, bwImage, CV_BGR2GRAY);
GaussianBlur(bwImage, processedImage, Size(9, 9), 9);
threshold(processedImage, processedImage, 25, 255, THRESH_BINARY); //THRESH_OTSU
int numberContours = -1;
int iterations = 1;
imshow("Processed", processedImage);
}
vector<Vec3f> circles;
Mat element = getStructuringElement(MORPH_ELLIPSE, Size(5, 5));
float dp2 = dp;
while (true)
{
float dp2 = dp;
Mat circleImage = processedImage.clone();
origImg = testImage.clone();
if (iterations > 0) dilate(circleImage, circleImage, element, Point(-1, -1), iterations);
Mat cannyImage;
Canny(circleImage, cannyImage, 100, 20);
imshow("Canny", cannyImage);
HoughCircles(circleImage, circles, HOUGH_GRADIENT, dp2/10, 5, houghParam1, houghParam2, 300, 5000);
cvtColor(circleImage, circleImage, CV_GRAY2BGR);
for (size_t i = 0; i < circles.size(); i++)
{
Scalar color = Scalar(0, 0, 255);
Point center2(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius2 = cvRound(circles[i][2]);
if (abs(center2.x - x) < 10 && abs((center2.y - y) < 10) && abs(radius - radius2) < 20) color = Scalar(0, 255, 0);
circle(circleImage, center2, 3, color, -1, 8, 0);
circle(circleImage, center2, radius2, color, 3, 8, 0);
circle(origImg, center2, 3, color, -1, 8, 0);
circle(origImg, center2, radius2,color, 3, 8, 0);
}
//Manual circles
circle(circleImage, Point(x, y), 3, Scalar(128, 0, 128), -1, 8, 0);
circle(circleImage, Point(x, y), radius, Scalar(128, 0, 128), 3, 8, 0);
circle(origImg, Point(x, y), 3, Scalar(128, 0, 128), -1, 8, 0);
circle(origImg, Point(x, y), radius, Scalar(128, 0, 128), 3, 8, 0);
imshow("Circles", circleImage);
imshow("Circled Orig", origImg);
int x = waitKey(50);
}
Mat drawnImage;
cvtColor(processedImage, drawnImage, CV_GRAY2BGR);
return 1;
}
Thanks #jalconvolvon - this is an interesting problem. Here's my result:
What I find important on and on is using dynamic parameter adjustment when prototyping, thus I include the function I used to tune Canny detection. The code also uses this answer for the Ransac part.
import cv2
import numpy as np
import auxcv as aux
from skimage import measure, draw
def empty_function(*arg):
pass
# tune canny edge detection. accept with pressing "C"
def CannyTrackbar(img, win_name):
trackbar_name = win_name + "Trackbar"
cv2.namedWindow(win_name)
cv2.resizeWindow(win_name, 500,100)
cv2.createTrackbar("canny_th1", win_name, 0, 255, empty_function)
cv2.createTrackbar("canny_th2", win_name, 0, 255, empty_function)
cv2.createTrackbar("blur_size", win_name, 0, 255, empty_function)
cv2.createTrackbar("blur_amp", win_name, 0, 255, empty_function)
while True:
trackbar_pos1 = cv2.getTrackbarPos("canny_th1", win_name)
trackbar_pos2 = cv2.getTrackbarPos("canny_th2", win_name)
trackbar_pos3 = cv2.getTrackbarPos("blur_size", win_name)
trackbar_pos4 = cv2.getTrackbarPos("blur_amp", win_name)
img_blurred = cv2.GaussianBlur(img.copy(), (trackbar_pos3 * 2 + 1, trackbar_pos3 * 2 + 1), trackbar_pos4)
canny = cv2.Canny(img_blurred, trackbar_pos1, trackbar_pos2)
cv2.imshow(win_name, canny)
key = cv2.waitKey(1) & 0xFF
if key == ord("c"):
break
cv2.destroyAllWindows()
return canny
img = cv2.imread("sphere.jpg")
#resize for convenience
img = cv2.resize(img, None, fx = 0.2, fy = 0.2)
#closing
kernel = np.ones((11,11), np.uint8)
img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)
#sharpening
kernel = np.array([[-1,-1,-1], [-1,9,-1], [-1,-1,-1]])
img = cv2.filter2D(img, -1, kernel)
#test if you use different scale img than 0.2 of the original that I used
#remember that the actual kernel size for GaussianBlur is trackbar_pos3*2+1
#you want to get as full circle as possible here
#canny = CannyTrackbar(img, "canny_trakbar")
#additional blurring to reduce the offset toward brighter region
img_blurred = cv2.GaussianBlur(img.copy(), (8*2+1,8*2+1), 1)
#detect edge. important: make sure this works well with CannyTrackbar()
canny = cv2.Canny(img_blurred, 160, 78)
coords = np.column_stack(np.nonzero(canny))
model, inliers = measure.ransac(coords, measure.CircleModel,
min_samples=3, residual_threshold=1,
max_trials=1000)
rr, cc = draw.circle_perimeter(int(model.params[0]),
int(model.params[1]),
int(model.params[2]),
shape=img.shape)
img[rr, cc] = 1
import matplotlib.pyplot as plt
plt.imshow(img, cmap='gray')
plt.scatter(model.params[1], model.params[0], s=50, c='red')
plt.axis('off')
plt.savefig('sphere_center.png', bbox_inches='tight')
plt.show()
Now I'd probably try to calculate where pixels are statisticaly brigher and where they are dimmer to adjust the laser position (if I understand correctly what you're trying to do)
If the Ransac is still not enough. I'd try tuning Canny to only detect a perfect arc on top of the circle (where it's well outlined) and than try using the following dependencies (I suspect that this should be possible):
I have a very basic code that uses the standardized HoughCircles command in openCV to detect a circle. However, my problem is that my data (images) are generated using an algorithm (for the purpose of data simulation) that plots a point in the range of +-15% (randomly in this range) of r (where r is the radius of the circle, that has been randomly generated to be between 5 and 10 (real numbers)) and does so for 360 degrees using the equation of a circle. (Attached a sample image).
http://imgur.com/a/iIZ1N
Now using the Hough circle command, I was able to detect a circle of approximately the same radius by manually playing around with the parameters (by settings up trackbars, inspired from a github code of the same nature) but I want to automate the process as I have over a 1000 images that I want to do this over and over on. Is there a better way to do that? Or if anyone has any suggestions, I would highly appreciate them as I am a beginner in the field of image processing and have a physics background rather than a CS one.
A rough sample of my code (without trackbars etc is below):
Mat img = imread("C:\\Users\\walee\\Documents\\MATLAB\\plot_2 .jpeg", 0);
Mat cimg,copy;
copy = img;
medianBlur(img, img, 5);
GaussianBlur(img, img, Size(1, 5), 1.1, 0);
cvtColor(img, cimg, COLOR_GRAY2BGR);
vector<Vec3f> circles;
HoughCircles(img, circles, HOUGH_GRADIENT,1, 10, 94, 57, 120, 250);
for (size_t i = 0; i < circles.size(); i++)
{
Vec3i c = circles[i];
circle(cimg, Point(c[0], c[1]), c[2], Scalar(0, 0, 255), 1, LINE_AA);
circle(cimg, Point(c[0], c[1]), 2, Scalar(0, 255, 0), 1, LINE_AA);
}
imshow("detected circles", cimg);
waitKey();
return 0;
If all images have the same nature (black axis and points as circles) I would suggest to do following:
1) remove axis by finding black elements and replace them with background
2) invert colours to have black background
3) perform morphological closing to fill the circles and create more solid points
4) (optional) if the density of the points is high you can try to apply another morphological operation, namely dilation to make the data circle thinner
5) apply Hough circle
(editing the question completely as i am trying a different approach)
I am trying to find the corners of a plain paper sheet which can be partially obstructed.
I am using houghlines now to get the lines and then calculating their intersections, this approach is robust and gives good results even if the paper is partially obstructed.
Problems:
1) There are multiple intersection points very closeby, how do i merge them fast as i am displaying the results in realtime.
2) As there might be some points detected outside the paper (due to some noise of some other object) how do i get a polygon from the intersection points.
GaussianBlur(grayscaleMat, grayscaleMat, cvSize(11,11), 0);//change from median blur to gaussian for more accuracy of square detection
cv::Canny(grayscaleMat, grayscaleMat, 50, 150, 3);
cv::Mat color;
cv::cvtColor(grayscaleMat, color, cv::COLOR_GRAY2BGR);
vector<cv::Vec2f> lines;
HoughLines(grayscaleMat, lines, 1, CV_PI/180, 50, 0, 0 );
if(lines.size() == 0) {
return image;
}
////////////////////////////////////
// Start geometery
//merge close lines
std::vector<cv::Vec2f> linesMerged = [ATOpenCVGeometery mergeCloseLines:lines];
//find intersections
std::vector<cv::Point2f> points = [ATOpenCVGeometery intersectionsBetweenLines:linesMerged];
for(int i = 0;i < linesMerged.size();i++) {
[self drawLine:image vec2f:linesMerged[i] color:cv::Scalar(0,0,255)];
}
for(int i = 0;i < points.size();i++) {
cv::circle(image, points[i], 4, cv::Scalar(255,0,0));
}
Given the following (canny'd) image, I'd like to grab the start/end endpoints of the full upper horizontal line.
I've tried opencv's HoughLineP function, but get segments rather than a full line. I realise that this is due to the camera calibration distortion.
Is there some other technique that is more forgiving when it comes to curvy distortions?
How does the theta parameter (HoughLineP function) work?
Alternatively, what would be a good way to join points that close to each other (with somehow similar angle)
Original:
Code:
Mat scene = imread("houghLines.png", 0);
vector<Vec4i> lines;
HoughLinesP(scene, lines, 1, CV_PI/180, 40, 100, 20 );
cvtColor(scene, scene, COLOR_GRAY2BGR); scene *= 0.5; // convert to colour
auto colours = generateColours((int)lines.size());
for(int i = 0; i < lines.size(); i++) {
auto l = lines[i];
line(scene, Point(l[0], l[1]), Point(l[2], l[3]), colours[i], 1, CV_AA);
}
imshow("scene", scene);
imwrite(getTempFilename(), scene);
waitKey();
Result: